-- Logs begin at Tue 2022-09-06 20:31:42 UTC, end at Tue 2022-09-06 21:36:53 UTC. -- Sep 06 20:58:52.383554 dv-cinder-749155-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 06 20:58:52.395018 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 6 20:58:52 2022] *** Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: nodename: dv-cinder-749155-11-iscsi Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: machine: x86_64 Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: clock source: unix Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: pcre jit disabled Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: detected number of CPU cores: 4 Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: current working directory: / Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: detected binary path: /usr/bin/uwsgi-core Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: your processes number limit is 47680 Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: your memory page size is 4096 bytes Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: detected max file descriptor number: 2048 Sep 06 20:58:52.397158 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: lock engine: pthread robust mutexes Sep 06 20:58:52.397889 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: thunder lock: enabled Sep 06 20:58:52.397889 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 06 20:58:52.397889 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 06 20:58:52.435819 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: Python main interpreter initialized at 0x564c5300e9c0 Sep 06 20:58:52.435819 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: python threads support enabled Sep 06 20:58:52.435819 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: your server socket listen backlog is limited to 100 connections Sep 06 20:58:52.435819 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: your mercy for graceful operations on workers is 90 seconds Sep 06 20:58:52.436245 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: mapped 403077 bytes (393 KB) for 2 cores Sep 06 20:58:52.436245 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: *** Operational MODE: preforking *** Sep 06 20:58:52.436549 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: *** uWSGI is running in multiple interpreter mode *** Sep 06 20:58:52.436549 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: spawned uWSGI master process (pid: 81683) Sep 06 20:58:52.436748 dv-cinder-749155-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 06 20:58:52.438646 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: spawned uWSGI worker 1 (pid: 81697, cores: 1) Sep 06 20:58:52.438646 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: spawned uWSGI worker 2 (pid: 81698, cores: 1) Sep 06 20:58:52.438646 dv-cinder-749155-11-iscsi devstack@keystone.service[81683]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 06 20:58:56.739657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 20:58:56.739657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: from cryptography.utils import int_from_bytes Sep 06 20:58:56.740501 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 20:58:56.740501 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: from cryptography.utils import int_from_bytes Sep 06 20:58:56.740987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 20:58:56.740987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: from cryptography.utils import int_from_bytes Sep 06 20:58:56.741122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 20:58:56.741122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: from cryptography.utils import int_from_bytes Sep 06 20:58:56.790055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 20:58:56.790269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 20:58:56.790406 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] command line args: [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 20:58:56.790505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 20:58:56.790708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 20:58:56.790841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 20:58:56.790922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ================================================================================ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 20:58:56.790988 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] command line args: [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 20:58:56.791127 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] admin_token = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.791275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 20:58:56.791275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ================================================================================ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 20:58:56.791410 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config_dir = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.791601 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.791733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] admin_token = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.791824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config_source = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.791892 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config_dir = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.791974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] debug = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.792036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.792098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.792369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config_source = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.792369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] debug = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.792478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.792539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.792627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.792733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.792889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.792966 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.793082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.793155 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.793438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.793533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.793595 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_config_append = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.793651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.793705 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.793760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_config_append = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.793821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_dir = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.793964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.794044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.794108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_dir = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.794215 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.794292 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.794598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.794769 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.794876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.794980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.795075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.795180 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.795282 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.795385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.795493 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.795588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.795681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.795780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.795881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.795983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.796111 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.796232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.796332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.796425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.796563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.796729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.796832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.796941 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.797051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.797150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.797252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.797347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.797536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.797700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.797824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.798055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.79/identity {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.798227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] publish_errors = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.798332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.79/identity {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.798436 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.798540 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] publish_errors = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.798640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.798735 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.798828 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.798926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.799029 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.799154 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.799288 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.799399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.799521 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] standard_threads = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.799633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.799742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.799857 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] standard_threads = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.799970 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.800082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.800243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.800363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.800473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_journal = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.800587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.800719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_json = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.800844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_journal = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.800966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_stderr = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.801079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_json = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.801196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_syslog = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.801307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_stderr = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.801569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.801690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_syslog = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.801852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.802022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 20:58:56.802186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.802310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.802423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.802540 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.802646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.802646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.802861 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.802973 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.803090 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.803208 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.803333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.803445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.803561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.803680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.803792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.803908 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.804031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.804143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.804253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.804368 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.804479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.804594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.804735 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.804856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.804968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.805077 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.805186 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.805296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.805416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.805528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.805651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.805762 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.805872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.805988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.806184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.806306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.806426 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.806538 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.806654 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.806771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.806894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.807006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.807122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.807233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.807343 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.807454 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.807573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.807685 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.807795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.807905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.808028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.808152 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.808265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.808375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.808487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.808597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.808762 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.808887 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.809006 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.809125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.809243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.809354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.809464 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.809583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.809695 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.809804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.809920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.810037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.810149 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.810266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.810378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.810487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.810597 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.810710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.810828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.810955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.811074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.811185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.811295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.811405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.811515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.811706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.811826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.811937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.812046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.812504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.812618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.812756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.812870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.812980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.813089 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.813207 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.813326 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.813447 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.813560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.813676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.813789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.813883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.813976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.814076 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.814177 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.814272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.814365 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.814457 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.814549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.814650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.814746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.814838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.814930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.815047 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.815190 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.815315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.815428 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.815544 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.815656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.815766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.815886 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.815998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.816109 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.816219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.816330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.816441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.816564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.816695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.816828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.816943 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.817063 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.817175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.817287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.817397 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.817508 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.817627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.817745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.817857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.817968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.818079 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.818199 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.818318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.818429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.818540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.818651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.818761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.818880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.818985 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.819103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.819215 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.819325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.819436 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.819561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.819676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.819797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.819908 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.820027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.820138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.820258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.820376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.820489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.820605 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.820746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.820859 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.820985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.821106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.821106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.821317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.821427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.821537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.821655 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.821767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.821767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.821975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.822086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.822196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.822315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.822427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.822427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.822633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.822744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.822854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.823044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.823159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.823270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.823380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.823490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.823608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.823729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.823850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.823977 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.824089 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.824201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.824311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.824422 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.824532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.824665 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.824775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.824886 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.824997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.825108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.825218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.825342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.825462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.825574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.825749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.825866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.825990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.826104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.826215 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.826325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.826436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.826546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.826664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.826785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.external = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.826899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.827040 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.827165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.827280 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.827395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.827506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.827628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.external = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.827741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.password = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.827853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.827965 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.token = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.827965 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.828180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.828180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.828409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.828523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.password = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.828634 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.828790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.token = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.828902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.829027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.829142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.829254 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.829366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.829478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.829588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.829712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.829826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.829938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.830050 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.830161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.830273 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.830393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.830507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.830618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.830733 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.830845 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.830962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.831084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.831199 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.831311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.831431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.831431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.831647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.831771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.831886 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.831886 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.832097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.832209 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.832321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.832442 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.832555 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.832682 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.832796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.832921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.833035 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.833156 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.833270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.833270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.833479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.833590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.833703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.833823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.833936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.834048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.834159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.834271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.834382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.834505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.834620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.834733 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.834844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.834956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.835067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.835187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.835307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.835420 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.835532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.835644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.835756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.835877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.835991 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.836112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.836224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.836336 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.836528 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.836644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.836829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.836941 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.837053 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.837165 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.837291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.837412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.837525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.837637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.837790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.837914 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.838036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.838156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.838270 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.838382 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.838494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.838619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.838740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.838854 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.838968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.839065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.839159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.839254 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.839254 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.839441 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.839535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.839629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.839732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.839846 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.839951 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.840047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.840141 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.840258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.840371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.840483 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.840603 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.840746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.840861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.840975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.841088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.841200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.841321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.841435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.841547 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.841641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.841718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.841832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.841953 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.842067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.842179 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.842291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.842402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.842513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.842633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.842747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.842858 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.842970 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.843082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.843200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.843322 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.843436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.843548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.843660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.843771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.843885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.844005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.844119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.844231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.844342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.844453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.844565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.844718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.844836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.844948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.845060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.845172 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.845284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.845443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.845559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.845671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.845783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.845895 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.846013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.846169 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.846286 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.846398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.846510 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.846622 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.846733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.846853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.846967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.847079 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.847191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.847304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.847423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.847545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.847662 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.847773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.847885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.847997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.848117 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.848231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.848343 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.848455 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.848566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.848705 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.848830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.848944 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.849056 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.849168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.849280 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.849393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.849512 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.849626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.849738 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.849849 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.849849 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.850058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.850178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.850291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.850404 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.850521 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.850634 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.850749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.850870 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.850983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.851095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.851206 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.851317 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.851428 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.851548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.851662 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.851773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.851885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.851996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.852107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.852227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.852340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.852454 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.852566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.852694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.852808 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.852933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.853047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.853158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.853269 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.853380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.853491 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.853612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.853725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.853837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.853948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.854059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.854170 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.854290 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.854404 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.854515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.854628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.854739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.854851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.854971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.855085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.855196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.855307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.855418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.855529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.855651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.855764 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.855875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.855987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.856098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.856210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.856333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.856447 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.856559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.856717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.856833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.857030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.857162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.857277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.857390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.857503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.857615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.857727 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.857850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.857850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.858062 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.858174 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.858287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.858399 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.858586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.858707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.858822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.858934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.859045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.859157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.859278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.859391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.859503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.859615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.859726 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.859839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.859959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.860073 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.860185 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.860296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.860408 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.860530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.860644 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.860779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.860894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.861006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.861118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.861242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.861358 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.861469 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.861581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.861693 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.861805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.861925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.862040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.862151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.862262 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.862373 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.862484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.862604 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.862718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.862831 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.862942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.863055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.863167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.863287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.863402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.863514 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.863625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.863797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.863913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.864035 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.864150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.864263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.864374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.864486 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.864597 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.864751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.864868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.864979 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.865092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.865203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.865323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.865437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.865548 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.865660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.865772 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.865883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.866003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.866116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.866229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.866355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.866468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.866580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.866700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.866814 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.866925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.867037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.867148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.867261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.867380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.867494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.867606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.867717 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.867829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.867940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.868084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.868201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.868315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.868432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.868544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.868711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.868840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.868955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.869067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.869178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.869292 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.869403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.869523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.869637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.869748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.869867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.869979 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.870090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.870447 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.870593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.870724 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.870839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.870839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.871055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.871168 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.871279 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.871401 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.871515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.871626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.871737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.871737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.871954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.872069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.872180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.872301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.872465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.872581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.872713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.872713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.872940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.873054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.873166 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.873278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.873388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.873502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.873648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.873763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.873875 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.873987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.874100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.874212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.874334 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.874448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.874561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.874673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.874785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.874897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.875018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.875132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.875244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.875355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.875467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 20:58:56.875581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.875756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.876025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.876243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.876474 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.876738 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.876977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 20:58:56.877149 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 20:58:56.905176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:56.905363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:56.905598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:56.905746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:56.905921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:56.906150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:56.913990 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:56.914614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:56.943089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:56.943281 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:56.943598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:56.943830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:56.996609 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:56.997186 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.045314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.045707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.047049 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.047287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.206388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.206575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.206720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.206810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.206873 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.206996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.207073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.207146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.207257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.207346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.207505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.207714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.231647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.231981 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.232087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.232211 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.232291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.232354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.232496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.232576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.232637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.232776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.232892 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.232983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.233058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.233227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.244265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.244597 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.257091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.257299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.257548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.257673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.257813 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.257917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.258034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.258115 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.258176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.258294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.258379 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.258452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.258570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.258777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 20:58:57.275148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.275329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 20:58:57.275693 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.275816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.275954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 20:58:57.276034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 20:58:57.276128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.276215 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.276415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.276536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 20:58:57.276694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.276776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.276954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.277093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.277230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.277432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.286589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.286864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.287133 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.287280 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.287392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.287483 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.287689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.287974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.294777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.295218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.297704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.297985 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.298153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.298269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.298444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.298536 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.298750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.298875 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.299011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.299112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.299283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.299408 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.299526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.299617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.299815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.299939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.300088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.300213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.300331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.300436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.300757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.301034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.323795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.324388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.325371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.325697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.326049 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.326342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.330385 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.330670 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.330947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.331069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.331320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.331606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.337671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.337955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.338078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.338297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.343686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.343948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.344148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.344266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.344400 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.344502 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.344706 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.344798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.345020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.345146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.345281 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.345361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.345562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.345702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.345877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.346093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.361626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.361860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.361984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.362141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.365764 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.366047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.369049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.369319 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.369464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.369604 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.373696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.374044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.376900 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.377216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.380099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.380395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.380587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.380711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.380897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.380992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.381206 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.381331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.381474 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.381567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.381769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.382042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.397671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.397959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.398101 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.398248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.398366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.398455 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.398657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.398929 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.405456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.406025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.407006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.407670 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.408533 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.408850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.409257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.409543 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.411588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.411869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.412149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.412286 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.412424 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.412504 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.412779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.412935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.413059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.413508 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.424097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.424382 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.424695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.424927 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.425053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.425191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.425308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.425493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.425618 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.425758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.425863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.426142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.426458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.426734 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.441780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.442695 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.444698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.445616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.447570 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.448618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.449106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.450096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.450688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.450969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.451356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.451643 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.451931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.452332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.457836 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.458778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.459240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.460256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.462163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.462561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.462845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.463117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.463240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.463582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.463926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.464376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.471074 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.471366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.471653 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.472416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.472746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.473030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.478546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.478828 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.479105 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.479678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.479678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.479899 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.480173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.480298 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 20:58:57.480490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.480565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.480704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.480911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.481064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.481157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.481318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.481402 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.481510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.481706 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.481866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.482015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.482097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.482289 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.482573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.482852 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.483137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.483406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.483678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.483936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 20:58:57.516445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564c5300e9c0 pid: 81698 (default app) Sep 06 20:58:57.518639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564c5300e9c0 pid: 81697 (default app) Sep 06 20:58:57.519864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421ef28-349b-44ff-a83a-d0a4dc768f51 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:58:57.520147 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421ef28-349b-44ff-a83a-d0a4dc768f51 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:58:57.520414 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421ef28-349b-44ff-a83a-d0a4dc768f51 None None] PATH_INFO: `/v3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:58:57.524629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1/1] 10.222.0.79 () {50 vars in 791 bytes} [Tue Sep 6 20:58:57 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:04.400745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62108754-2dea-4004-94ba-520b7b9c0ca1 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:04.401325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62108754-2dea-4004-94ba-520b7b9c0ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:04.401325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62108754-2dea-4004-94ba-520b7b9c0ca1 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:04.405720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1/2] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:59:04.412636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dec291-938d-4db7-8542-c8db96ef2264 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:04.413605 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dec291-938d-4db7-8542-c8db96ef2264 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:04.413605 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dec291-938d-4db7-8542-c8db96ef2264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:04.486087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: INFO dbcounter [None req-11dec291-938d-4db7-8542-c8db96ef2264 None None] Registered counter for database keystone Sep 06 20:59:04.507179 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG oslo_db.sqlalchemy.engines [None req-11dec291-938d-4db7-8542-c8db96ef2264 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81698) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 20:59:04.519532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writer thread running {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 20:59:04.706616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG passlib.handlers.bcrypt [None req-11dec291-938d-4db7-8542-c8db96ef2264 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81698) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 20:59:04.706915 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG passlib.handlers.bcrypt [None req-11dec291-938d-4db7-8542-c8db96ef2264 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81698) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 20:59:04.767861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-11dec291-938d-4db7-8542-c8db96ef2264 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:04.769557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-11dec291-938d-4db7-8542-c8db96ef2264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:04.912733 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 2/3] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 06 20:59:05.473201 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62108754-2dea-4004-94ba-520b7b9c0ca1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:05.551936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: INFO dbcounter [None req-62108754-2dea-4004-94ba-520b7b9c0ca1 None None] Registered counter for database keystone Sep 06 20:59:05.577484 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG oslo_db.sqlalchemy.engines [None req-62108754-2dea-4004-94ba-520b7b9c0ca1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81697) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 20:59:05.586193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writer thread running {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 20:59:05.705913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62108754-2dea-4004-94ba-520b7b9c0ca1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:05.711367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c638508-79a5-49a9-9726-b2ceaf6c530c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:05.711997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c638508-79a5-49a9-9726-b2ceaf6c530c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:05.712274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c638508-79a5-49a9-9726-b2ceaf6c530c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:05.712539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c638508-79a5-49a9-9726-b2ceaf6c530c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:05.721712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c638508-79a5-49a9-9726-b2ceaf6c530c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:05.774937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6c638508-79a5-49a9-9726-b2ceaf6c530c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 20:59:05.775270 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c638508-79a5-49a9-9726-b2ceaf6c530c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:05.785793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 2/4] 10.222.0.79 () {58 vars in 1324 bytes} [Tue Sep 6 20:59:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 20:59:05.796415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dec291-938d-4db7-8542-c8db96ef2264 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:05.820178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dec291-938d-4db7-8542-c8db96ef2264 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:05.827579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87929640-a297-45ac-96c1-c92cd4deecb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:05.828713 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87929640-a297-45ac-96c1-c92cd4deecb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:05.829020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87929640-a297-45ac-96c1-c92cd4deecb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:05.829289 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87929640-a297-45ac-96c1-c92cd4deecb3 admin admin] PATH_INFO: `/v3/projects/0b4b2ee27141432e8ca02dac223fb495` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:05.830975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87929640-a297-45ac-96c1-c92cd4deecb3 admin admin] RBAC: Authorizing `identity:get_project(project_id=0b4b2ee27141432e8ca02dac223fb495)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:05.884698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: WARNING keystone.common.rbac_enforcer.enforcer [None req-87929640-a297-45ac-96c1-c92cd4deecb3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 20:59:05.885057 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87929640-a297-45ac-96c1-c92cd4deecb3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:05.889794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 3/5] 10.222.0.79 () {56 vars in 1207 bytes} [Tue Sep 6 20:59:05 2022] GET /identity/v3/projects/0b4b2ee27141432e8ca02dac223fb495 => generated 344 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:08.038081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02487435-b6c8-44d7-bf49-65bb3c95c9f9 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:08.038648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02487435-b6c8-44d7-bf49-65bb3c95c9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:08.038648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02487435-b6c8-44d7-bf49-65bb3c95c9f9 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:08.042833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 3/6] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:08.049672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12dd70-76fd-43a7-af0e-9204e2887722 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:08.050373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12dd70-76fd-43a7-af0e-9204e2887722 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:08.050528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12dd70-76fd-43a7-af0e-9204e2887722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:08.081954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3b12dd70-76fd-43a7-af0e-9204e2887722 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:08.083516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3b12dd70-76fd-43a7-af0e-9204e2887722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:08.092121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 4/7] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 06 20:59:08.355966 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02487435-b6c8-44d7-bf49-65bb3c95c9f9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:08.375060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02487435-b6c8-44d7-bf49-65bb3c95c9f9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:08.379579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aeef15-51f0-433a-8e8e-c96ba1696f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:08.380200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aeef15-51f0-433a-8e8e-c96ba1696f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:08.380480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aeef15-51f0-433a-8e8e-c96ba1696f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:08.380791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aeef15-51f0-433a-8e8e-c96ba1696f17 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:08.390029 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aeef15-51f0-433a-8e8e-c96ba1696f17 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:08.391847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aeef15-51f0-433a-8e8e-c96ba1696f17 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:08.401503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 4/8] 10.222.0.79 () {58 vars in 1324 bytes} [Tue Sep 6 20:59:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 20:59:08.409668 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12dd70-76fd-43a7-af0e-9204e2887722 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:08.425162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12dd70-76fd-43a7-af0e-9204e2887722 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:08.434816 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c20a90-875c-4819-9a43-1fc13adf7684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:08.435444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c20a90-875c-4819-9a43-1fc13adf7684 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:08.435722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c20a90-875c-4819-9a43-1fc13adf7684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:08.436144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c20a90-875c-4819-9a43-1fc13adf7684 admin admin] PATH_INFO: `/v3/users/42a19c5d3ad549648071d18e7300d6c1` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:08.437969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c20a90-875c-4819-9a43-1fc13adf7684 admin admin] RBAC: Authorizing `identity:get_user(user_id=42a19c5d3ad549648071d18e7300d6c1)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:08.439815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c20a90-875c-4819-9a43-1fc13adf7684 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:08.441452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 5/9] 10.222.0.79 () {56 vars in 1198 bytes} [Tue Sep 6 20:59:08 2022] GET /identity/v3/users/42a19c5d3ad549648071d18e7300d6c1 => generated 247 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:10.688121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94bb21-386c-4f56-beba-461fde2c4aae None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:10.688689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94bb21-386c-4f56-beba-461fde2c4aae None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:10.688689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94bb21-386c-4f56-beba-461fde2c4aae None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:10.690011 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 5/10] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:59:10.695725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e08a45-f795-4473-ac30-dde8e6c6e5cd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:10.696221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e08a45-f795-4473-ac30-dde8e6c6e5cd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:10.696311 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e08a45-f795-4473-ac30-dde8e6c6e5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:10.701332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe640df2-30c1-4703-9bab-cdfc56d59330 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:10.701610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe640df2-30c1-4703-9bab-cdfc56d59330 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:10.701882 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe640df2-30c1-4703-9bab-cdfc56d59330 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:10.703195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 6/11] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:59:10.708515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fcca6-6603-448a-857a-306318faa1cd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:10.708840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fcca6-6603-448a-857a-306318faa1cd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:10.710330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fcca6-6603-448a-857a-306318faa1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:10.716951 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-53e08a45-f795-4473-ac30-dde8e6c6e5cd None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:10.718159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-53e08a45-f795-4473-ac30-dde8e6c6e5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:10.752545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG passlib.handlers.bcrypt [None req-380fcca6-6603-448a-857a-306318faa1cd None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81697) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 20:59:10.753022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG passlib.handlers.bcrypt [None req-380fcca6-6603-448a-857a-306318faa1cd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81697) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 20:59:10.801394 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-380fcca6-6603-448a-857a-306318faa1cd None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:10.803076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-380fcca6-6603-448a-857a-306318faa1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:10.804978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 6/12] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:10.907698 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 7/13] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:11.071964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e08a45-f795-4473-ac30-dde8e6c6e5cd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:11.098411 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:11.168786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fcca6-6603-448a-857a-306318faa1cd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:11.182131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e08a45-f795-4473-ac30-dde8e6c6e5cd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:11.217791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d0735-275b-4366-b070-ad65bd8517ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:11.218438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0735-275b-4366-b070-ad65bd8517ec None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:11.218717 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0735-275b-4366-b070-ad65bd8517ec None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:11.218991 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0735-275b-4366-b070-ad65bd8517ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:11.231580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d0735-275b-4366-b070-ad65bd8517ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-749d0735-275b-4366-b070-ad65bd8517ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:11.240674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:11.241758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:11.242581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:11.242581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:11.242581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:11.242581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:11.242581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:11.242581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:11.242581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:11.242581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:11.242581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:11.242581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:11.242581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:11.242581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:11.242581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 7/14] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 20:59:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:11.249610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d0735-275b-4366-b070-ad65bd8517ec None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:11.291422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fcca6-6603-448a-857a-306318faa1cd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:11.294500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:11.334530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeb0a5-3a2f-4f10-b7fe-ea17ea1584d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:11.335519 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeb0a5-3a2f-4f10-b7fe-ea17ea1584d4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:11.335692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeb0a5-3a2f-4f10-b7fe-ea17ea1584d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:11.335952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeb0a5-3a2f-4f10-b7fe-ea17ea1584d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:11.340738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:11.363497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d0735-275b-4366-b070-ad65bd8517ec None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:11.400408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54ddc4-aa9e-4d6a-b713-419c0e37cbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:11.401158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54ddc4-aa9e-4d6a-b713-419c0e37cbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:11.401438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54ddc4-aa9e-4d6a-b713-419c0e37cbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:11.401712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54ddc4-aa9e-4d6a-b713-419c0e37cbb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:11.402391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54ddc4-aa9e-4d6a-b713-419c0e37cbb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:11.402694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54ddc4-aa9e-4d6a-b713-419c0e37cbb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:11.404253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54ddc4-aa9e-4d6a-b713-419c0e37cbb4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:11.415147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeb0a5-3a2f-4f10-b7fe-ea17ea1584d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:11.416722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeb0a5-3a2f-4f10-b7fe-ea17ea1584d4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:11.416841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 8/15] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:11.423925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54ddc4-aa9e-4d6a-b713-419c0e37cbb4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:11.529704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 8/16] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:11.533891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54ddc4-aa9e-4d6a-b713-419c0e37cbb4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:11.535676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeb0a5-3a2f-4f10-b7fe-ea17ea1584d4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:11.538562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:11.572978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411da0f-1589-48c0-97ca-2f137232b7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:11.574082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411da0f-1589-48c0-97ca-2f137232b7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:11.574524 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411da0f-1589-48c0-97ca-2f137232b7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:11.574975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411da0f-1589-48c0-97ca-2f137232b7b9 None admin] PATH_INFO: `/v3/users/42a19c5d3ad549648071d18e7300d6c1` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:11.576698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411da0f-1589-48c0-97ca-2f137232b7b9 None admin] RBAC: Authorizing `identity:get_user(user_id=42a19c5d3ad549648071d18e7300d6c1)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:11.578263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411da0f-1589-48c0-97ca-2f137232b7b9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:11.579909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 9/17] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:11 2022] GET /identity/v3/users/42a19c5d3ad549648071d18e7300d6c1 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:11.585848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411da0f-1589-48c0-97ca-2f137232b7b9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:11.602803 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:11.644443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeb0a5-3a2f-4f10-b7fe-ea17ea1584d4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:11.679254 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9f8dc-9140-4a17-843a-ec8bc77c2a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:11.680043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9f8dc-9140-4a17-843a-ec8bc77c2a77 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:11.680317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9f8dc-9140-4a17-843a-ec8bc77c2a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:11.680585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9f8dc-9140-4a17-843a-ec8bc77c2a77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:11.689331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411da0f-1589-48c0-97ca-2f137232b7b9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:11.699402 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9f8dc-9140-4a17-843a-ec8bc77c2a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:11.701034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9f8dc-9140-4a17-843a-ec8bc77c2a77 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-74e9f8dc-9140-4a17-843a-ec8bc77c2a77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:59:11.719310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:11.720313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:59:11.721332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:59:11.721332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:59:11.721332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:11.721332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 9/18] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:11.724029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38530d05-04e3-4b64-88d3-852d850ad95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:11.724704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38530d05-04e3-4b64-88d3-852d850ad95c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:11.724982 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38530d05-04e3-4b64-88d3-852d850ad95c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:11.725253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38530d05-04e3-4b64-88d3-852d850ad95c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:11.726447 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9f8dc-9140-4a17-843a-ec8bc77c2a77 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:11.726557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38530d05-04e3-4b64-88d3-852d850ad95c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:11.727923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38530d05-04e3-4b64-88d3-852d850ad95c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:11.729520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 10/19] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:11.735966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38530d05-04e3-4b64-88d3-852d850ad95c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:11.786795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:11.814407 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:11.831499 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9f8dc-9140-4a17-843a-ec8bc77c2a77 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:11.833988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38530d05-04e3-4b64-88d3-852d850ad95c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:11.866757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770518b-10c0-4de7-a760-b32d6cd6a9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:11.867379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770518b-10c0-4de7-a760-b32d6cd6a9ea None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:11.867658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770518b-10c0-4de7-a760-b32d6cd6a9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:11.867928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770518b-10c0-4de7-a760-b32d6cd6a9ea None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:11.868013 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94f2fd-5563-492b-bb88-9987a2f64c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:11.868575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770518b-10c0-4de7-a760-b32d6cd6a9ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:11.868704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94f2fd-5563-492b-bb88-9987a2f64c39 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:11.868904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770518b-10c0-4de7-a760-b32d6cd6a9ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:11.868991 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94f2fd-5563-492b-bb88-9987a2f64c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:11.869175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94f2fd-5563-492b-bb88-9987a2f64c39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:11.869926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94f2fd-5563-492b-bb88-9987a2f64c39 None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.domain.id=default, user.id=42a19c5d3ad549648071d18e7300d6c1) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:11.870230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94f2fd-5563-492b-bb88-9987a2f64c39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:11.870322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770518b-10c0-4de7-a760-b32d6cd6a9ea None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:11.871672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94f2fd-5563-492b-bb88-9987a2f64c39 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:11.878896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 11/20] 10.222.0.79 () {56 vars in 1322 bytes} [Tue Sep 6 20:59:11 2022] GET /identity/v3/role_assignments?user.id=42a19c5d3ad549648071d18e7300d6c1&scope.domain.id=default&role.id=3b30504991194002b41f8587d0851c10 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:11.889041 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 10/21] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 20:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:14.379932 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa147b5-c543-4ea4-8805-5cda921e2aec None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:14.380857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa147b5-c543-4ea4-8805-5cda921e2aec None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:14.381265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa147b5-c543-4ea4-8805-5cda921e2aec None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:14.382754 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 12/22] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:14.401973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51322da0-e4ae-4827-91de-6e7f8146f37c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:14.402819 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51322da0-e4ae-4827-91de-6e7f8146f37c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:14.403316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51322da0-e4ae-4827-91de-6e7f8146f37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:14.429770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-51322da0-e4ae-4827-91de-6e7f8146f37c None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:14.431233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-51322da0-e4ae-4827-91de-6e7f8146f37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:14.563756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 11/23] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:14.614876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e25d00-82fc-48e3-8908-6ef1cde51886 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:14.615160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e25d00-82fc-48e3-8908-6ef1cde51886 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:14.615512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e25d00-82fc-48e3-8908-6ef1cde51886 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:14.616988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 13/24] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:14.622268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1522f-18f6-4664-abb1-862df7b92a23 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:14.622565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1522f-18f6-4664-abb1-862df7b92a23 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:14.622840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1522f-18f6-4664-abb1-862df7b92a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:14.647568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-47c1522f-18f6-4664-abb1-862df7b92a23 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:14.648696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-47c1522f-18f6-4664-abb1-862df7b92a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:14.708113 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:14.747383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 12/25] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:14.800103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b0aa2-5884-4f94-950e-d338b693be7a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:14.800530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b0aa2-5884-4f94-950e-d338b693be7a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:14.800695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b0aa2-5884-4f94-950e-d338b693be7a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:14.802384 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 14/26] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:14.818667 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40558ab9-8957-45ea-9665-68711c7107b5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:14.819038 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40558ab9-8957-45ea-9665-68711c7107b5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:14.820833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40558ab9-8957-45ea-9665-68711c7107b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:14.849762 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-40558ab9-8957-45ea-9665-68711c7107b5 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:14.850975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-40558ab9-8957-45ea-9665-68711c7107b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:14.935226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 13/27] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:15.177976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b0aa2-5884-4f94-950e-d338b693be7a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:15.233165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40558ab9-8957-45ea-9665-68711c7107b5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:15.389183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b0aa2-5884-4f94-950e-d338b693be7a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:15.403374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40558ab9-8957-45ea-9665-68711c7107b5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:15.414097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:15.447572 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be75c49-efdb-4e76-98c7-f79df6604577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:15.448203 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be75c49-efdb-4e76-98c7-f79df6604577 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:15.448474 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be75c49-efdb-4e76-98c7-f79df6604577 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:15.448822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be75c49-efdb-4e76-98c7-f79df6604577 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:15.449587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be75c49-efdb-4e76-98c7-f79df6604577 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:15.451247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be75c49-efdb-4e76-98c7-f79df6604577 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:15.509770 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9320d2d-5278-42aa-98c9-b4dc0a4560e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:15.511314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9320d2d-5278-42aa-98c9-b4dc0a4560e9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:15.512108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9320d2d-5278-42aa-98c9-b4dc0a4560e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:15.512956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9320d2d-5278-42aa-98c9-b4dc0a4560e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:15.554052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9320d2d-5278-42aa-98c9-b4dc0a4560e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e9320d2d-5278-42aa-98c9-b4dc0a4560e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:15.573662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:15.575338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:15.576782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:15.576782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:15.576782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:15.576782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:15.576782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:15.576782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:15.576782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:15.576782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:15.576782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:15.576782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:15.576782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:15.576782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:15.592679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 14/28] 10.222.0.79 () {60 vars in 1131 bytes} [Tue Sep 6 20:59:15 2022] POST /identity/v3/roles => generated 219 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:59:15.592679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be75c49-efdb-4e76-98c7-f79df6604577 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:15.597428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 15/29] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 20:59:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:15.604078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9320d2d-5278-42aa-98c9-b4dc0a4560e9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:15.623635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:15.783179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be75c49-efdb-4e76-98c7-f79df6604577 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:15.822955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df252bfa-67b7-4267-823e-f07ce2cff299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:15.824008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df252bfa-67b7-4267-823e-f07ce2cff299 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:15.824475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df252bfa-67b7-4267-823e-f07ce2cff299 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:15.825020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df252bfa-67b7-4267-823e-f07ce2cff299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:15.838633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df252bfa-67b7-4267-823e-f07ce2cff299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:15.842230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df252bfa-67b7-4267-823e-f07ce2cff299 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-df252bfa-67b7-4267-823e-f07ce2cff299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:59:15.877425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:15.878415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:59:15.879211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:59:15.879211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:59:15.879211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:15.884437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 15/30] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:15.889763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a593ff-a283-4334-8503-b176f4f8f2e4 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:15.890328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a593ff-a283-4334-8503-b176f4f8f2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:15.890783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a593ff-a283-4334-8503-b176f4f8f2e4 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:15.892388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 16/31] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:15.896417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a593ff-a283-4334-8503-b176f4f8f2e4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:15.935372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:15.939022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9320d2d-5278-42aa-98c9-b4dc0a4560e9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:16.001667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc96e8-7813-4ed5-ae01-5004e4f2b83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:16.002339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc96e8-7813-4ed5-ae01-5004e4f2b83a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:16.002614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc96e8-7813-4ed5-ae01-5004e4f2b83a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:16.002892 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc96e8-7813-4ed5-ae01-5004e4f2b83a None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:16.003585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc96e8-7813-4ed5-ae01-5004e4f2b83a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:16.003888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc96e8-7813-4ed5-ae01-5004e4f2b83a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:16.005719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc96e8-7813-4ed5-ae01-5004e4f2b83a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:16.017114 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 16/32] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:16.022014 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da7563-d71f-4354-bbdc-f46e13dda7c5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:16.022303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da7563-d71f-4354-bbdc-f46e13dda7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:16.022584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da7563-d71f-4354-bbdc-f46e13dda7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:16.049306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e7da7563-d71f-4354-bbdc-f46e13dda7c5 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:16.050592 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e7da7563-d71f-4354-bbdc-f46e13dda7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:16.058543 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a593ff-a283-4334-8503-b176f4f8f2e4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:16.107347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03111321-d5cc-405c-8664-bd7e26657b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:16.108625 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03111321-d5cc-405c-8664-bd7e26657b19 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:16.109139 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03111321-d5cc-405c-8664-bd7e26657b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:16.110036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03111321-d5cc-405c-8664-bd7e26657b19 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:16.110900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03111321-d5cc-405c-8664-bd7e26657b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:16.111386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03111321-d5cc-405c-8664-bd7e26657b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:16.113295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03111321-d5cc-405c-8664-bd7e26657b19 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:16.126146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:16.130917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 17/33] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 20:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:16.132342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03111321-d5cc-405c-8664-bd7e26657b19 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:16.134596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 17/34] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:16.142305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da7563-d71f-4354-bbdc-f46e13dda7c5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:16.236710 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:16.271906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da7563-d71f-4354-bbdc-f46e13dda7c5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:16.275744 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03111321-d5cc-405c-8664-bd7e26657b19 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:16.313662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d57738-4b06-475b-aa33-d8bc92e35074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:16.314232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d57738-4b06-475b-aa33-d8bc92e35074 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:16.314779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d57738-4b06-475b-aa33-d8bc92e35074 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:16.315052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d57738-4b06-475b-aa33-d8bc92e35074 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:16.315709 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d57738-4b06-475b-aa33-d8bc92e35074 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:16.317494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d57738-4b06-475b-aa33-d8bc92e35074 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:16.335777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbff047-9a21-424a-bebb-6495210b1149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:16.336946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbff047-9a21-424a-bebb-6495210b1149 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:16.337437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbff047-9a21-424a-bebb-6495210b1149 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:16.337983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbff047-9a21-424a-bebb-6495210b1149 None admin] PATH_INFO: `/v3/users/42a19c5d3ad549648071d18e7300d6c1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:16.341256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbff047-9a21-424a-bebb-6495210b1149 None admin] RBAC: Authorizing `identity:get_user(user_id=42a19c5d3ad549648071d18e7300d6c1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:16.344025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbff047-9a21-424a-bebb-6495210b1149 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:16.347692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 18/35] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:16 2022] GET /identity/v3/users/42a19c5d3ad549648071d18e7300d6c1 => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:16.349853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9283c4-a861-48fb-9538-0c7d236031fb None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:16.350425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9283c4-a861-48fb-9538-0c7d236031fb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:16.350836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9283c4-a861-48fb-9538-0c7d236031fb None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:16.353450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 19/36] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:16.354789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9283c4-a861-48fb-9538-0c7d236031fb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:16.483818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 18/37] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 20:59:16 2022] POST /identity/v3/projects => generated 301 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:59:16.490512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524b7eb-9860-446c-8560-15643746ebe1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:16.490954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524b7eb-9860-446c-8560-15643746ebe1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:16.491380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524b7eb-9860-446c-8560-15643746ebe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:16.517470 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3524b7eb-9860-446c-8560-15643746ebe1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:16.520090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3524b7eb-9860-446c-8560-15643746ebe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:16.550289 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9283c4-a861-48fb-9538-0c7d236031fb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:16.589563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:16.601568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8846d-bb85-48f9-b44b-dd18b8913e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:16.602219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8846d-bb85-48f9-b44b-dd18b8913e48 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:16.602493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8846d-bb85-48f9-b44b-dd18b8913e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:16.602925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8846d-bb85-48f9-b44b-dd18b8913e48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:16.604088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8846d-bb85-48f9-b44b-dd18b8913e48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:16.605685 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8846d-bb85-48f9-b44b-dd18b8913e48 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:16.612000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 20/38] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:16.612000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8846d-bb85-48f9-b44b-dd18b8913e48 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:16.624708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 19/39] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:16.626077 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfb384-dfce-4296-9114-7c4bbc9dfaaa None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:16.626367 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfb384-dfce-4296-9114-7c4bbc9dfaaa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:16.626644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfb384-dfce-4296-9114-7c4bbc9dfaaa None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:16.628013 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 20/40] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:16.628863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfb384-dfce-4296-9114-7c4bbc9dfaaa None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:16.633905 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:16.748813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8846d-bb85-48f9-b44b-dd18b8913e48 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:16.757802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfb384-dfce-4296-9114-7c4bbc9dfaaa None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:16.787880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5227d89-9aab-42e9-bf59-7b846cc15b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:16.788505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5227d89-9aab-42e9-bf59-7b846cc15b0b None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:16.788824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5227d89-9aab-42e9-bf59-7b846cc15b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:16.789101 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5227d89-9aab-42e9-bf59-7b846cc15b0b None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:16.789783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5227d89-9aab-42e9-bf59-7b846cc15b0b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:16.791184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5227d89-9aab-42e9-bf59-7b846cc15b0b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:16.795683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a35c6-3f91-4019-bfe9-0aa2747b2169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:16.796680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a35c6-3f91-4019-bfe9-0aa2747b2169 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:16.796974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a35c6-3f91-4019-bfe9-0aa2747b2169 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:16.797314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a35c6-3f91-4019-bfe9-0aa2747b2169 None admin] PATH_INFO: `/v3/domains/default/users/42a19c5d3ad549648071d18e7300d6c1/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:16.799555 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a35c6-3f91-4019-bfe9-0aa2747b2169 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=42a19c5d3ad549648071d18e7300d6c1, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:16.801517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a35c6-3f91-4019-bfe9-0aa2747b2169 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:16.812702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 21/41] 10.222.0.79 () {60 vars in 1131 bytes} [Tue Sep 6 20:59:16 2022] POST /identity/v3/roles => generated 225 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:59:16.812702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e5461-4da6-4f3a-8a6f-296783dbc3c7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:16.812702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e5461-4da6-4f3a-8a6f-296783dbc3c7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:16.812702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e5461-4da6-4f3a-8a6f-296783dbc3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:16.826527 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 21/42] 10.222.0.79 () {58 vars in 1361 bytes} [Tue Sep 6 20:59:16 2022] PUT /identity/v3/domains/default/users/42a19c5d3ad549648071d18e7300d6c1/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:59:16.836688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-618e5461-4da6-4f3a-8a6f-296783dbc3c7 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:16.836688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-618e5461-4da6-4f3a-8a6f-296783dbc3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:16.901680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a35c6-3f91-4019-bfe9-0aa2747b2169 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:16.927704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 22/43] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:16.957060 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:17.034667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a35c6-3f91-4019-bfe9-0aa2747b2169 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:17.072766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b4181-f94a-4122-8b2c-5553f0a4d55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:17.073431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b4181-f94a-4122-8b2c-5553f0a4d55c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:17.073714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b4181-f94a-4122-8b2c-5553f0a4d55c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:17.073988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b4181-f94a-4122-8b2c-5553f0a4d55c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:17.075215 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b4181-f94a-4122-8b2c-5553f0a4d55c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:17.076811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b4181-f94a-4122-8b2c-5553f0a4d55c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:17.078667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 22/44] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:17.084066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e5461-4da6-4f3a-8a6f-296783dbc3c7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:17.087812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:17.190142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e5461-4da6-4f3a-8a6f-296783dbc3c7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:17.208922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b4181-f94a-4122-8b2c-5553f0a4d55c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:17.230296 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f8037-72fd-4e11-b5c3-42c8c0893ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:17.230924 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f8037-72fd-4e11-b5c3-42c8c0893ade None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:17.231258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f8037-72fd-4e11-b5c3-42c8c0893ade None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:17.231538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f8037-72fd-4e11-b5c3-42c8c0893ade None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:17.232407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f8037-72fd-4e11-b5c3-42c8c0893ade None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:17.233714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f8037-72fd-4e11-b5c3-42c8c0893ade None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:17.376716 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 23/45] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 20:59:17 2022] POST /identity/v3/projects => generated 312 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:59:17.448838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b4181-f94a-4122-8b2c-5553f0a4d55c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:17.458083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:17.494641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a957902-e0a0-4c66-a9bd-2adbbb528ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:17.495306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a957902-e0a0-4c66-a9bd-2adbbb528ff6 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:17.495557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a957902-e0a0-4c66-a9bd-2adbbb528ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:17.495833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a957902-e0a0-4c66-a9bd-2adbbb528ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:17.496531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a957902-e0a0-4c66-a9bd-2adbbb528ff6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:17.498923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a957902-e0a0-4c66-a9bd-2adbbb528ff6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:17.519324 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 23/46] 10.222.0.79 () {60 vars in 1131 bytes} [Tue Sep 6 20:59:17 2022] POST /identity/v3/roles => generated 223 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:59:19.044600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddb3d5-b252-4e7c-909e-83c26438eb19 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:19.045175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddb3d5-b252-4e7c-909e-83c26438eb19 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:19.045175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddb3d5-b252-4e7c-909e-83c26438eb19 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:19.046528 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 24/47] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:19.051991 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365612b-0b25-4d1b-b26d-af6c2ce85706 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:19.052433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365612b-0b25-4d1b-b26d-af6c2ce85706 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:19.052513 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365612b-0b25-4d1b-b26d-af6c2ce85706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:19.071599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a365612b-0b25-4d1b-b26d-af6c2ce85706 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:19.072796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a365612b-0b25-4d1b-b26d-af6c2ce85706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:19.141628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 24/48] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:19.408076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddb3d5-b252-4e7c-909e-83c26438eb19 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:19.472516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:19.521568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddb3d5-b252-4e7c-909e-83c26438eb19 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:19.546347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94835e49-673d-42ce-8a4f-0663e223a551 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:19.546605 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94835e49-673d-42ce-8a4f-0663e223a551 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:19.546949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94835e49-673d-42ce-8a4f-0663e223a551 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:19.549089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 25/49] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:59:19.556754 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85444e70-831b-4785-8240-055ed5a367c3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:19.557126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85444e70-831b-4785-8240-055ed5a367c3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:19.557519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85444e70-831b-4785-8240-055ed5a367c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:19.562542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1609666-9898-4be7-9f7f-13da0b6fd706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:19.563193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1609666-9898-4be7-9f7f-13da0b6fd706 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:19.563562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1609666-9898-4be7-9f7f-13da0b6fd706 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:19.563834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1609666-9898-4be7-9f7f-13da0b6fd706 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:19.576270 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1609666-9898-4be7-9f7f-13da0b6fd706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:19.577286 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-85444e70-831b-4785-8240-055ed5a367c3 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:19.578499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-85444e70-831b-4785-8240-055ed5a367c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b1609666-9898-4be7-9f7f-13da0b6fd706 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:19.582721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:19.583580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:19.584341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:19.584341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:19.584341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:19.584341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:19.584341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:19.584341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:19.584341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:19.584341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:19.584341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:19.584341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:19.584341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:19.584341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:19.584341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 25/50] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 20:59:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:19.590801 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1609666-9898-4be7-9f7f-13da0b6fd706 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:19.653578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 26/51] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:19.691352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1609666-9898-4be7-9f7f-13da0b6fd706 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:19.707083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:19.727025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8d3a7-8544-4556-b2a2-3668713d3111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:19.727644 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8d3a7-8544-4556-b2a2-3668713d3111 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:19.727976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8d3a7-8544-4556-b2a2-3668713d3111 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:19.728195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8d3a7-8544-4556-b2a2-3668713d3111 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:19.728992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8d3a7-8544-4556-b2a2-3668713d3111 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:19.729749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8d3a7-8544-4556-b2a2-3668713d3111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:19.730693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8d3a7-8544-4556-b2a2-3668713d3111 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:19.742635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 26/52] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:19.748782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85444e70-831b-4785-8240-055ed5a367c3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:19.780515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:19.860570 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85444e70-831b-4785-8240-055ed5a367c3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:19.900892 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460120c-34b9-4dd6-829b-389c92af7fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:19.901514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460120c-34b9-4dd6-829b-389c92af7fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:19.901795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460120c-34b9-4dd6-829b-389c92af7fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:19.902070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460120c-34b9-4dd6-829b-389c92af7fe9 None admin] PATH_INFO: `/v3/users/42a19c5d3ad549648071d18e7300d6c1` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:19.903238 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460120c-34b9-4dd6-829b-389c92af7fe9 None admin] RBAC: Authorizing `identity:get_user(user_id=42a19c5d3ad549648071d18e7300d6c1)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:19.904758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460120c-34b9-4dd6-829b-389c92af7fe9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:19.906462 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 27/53] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:19 2022] GET /identity/v3/users/42a19c5d3ad549648071d18e7300d6c1 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:19.912189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8d3a7-8544-4556-b2a2-3668713d3111 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:19.915969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460120c-34b9-4dd6-829b-389c92af7fe9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:20.022635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:20.029191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8d3a7-8544-4556-b2a2-3668713d3111 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:20.030831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460120c-34b9-4dd6-829b-389c92af7fe9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:20.063874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1592a-cd24-43b7-8537-cff4e44f8e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:20.064599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1592a-cd24-43b7-8537-cff4e44f8e5c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:20.064907 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1592a-cd24-43b7-8537-cff4e44f8e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:20.065174 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1592a-cd24-43b7-8537-cff4e44f8e5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:20.066023 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481b922-21f9-45cd-bc2e-464cdaeed2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:20.066396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1592a-cd24-43b7-8537-cff4e44f8e5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:20.066669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481b922-21f9-45cd-bc2e-464cdaeed2eb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:20.066951 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481b922-21f9-45cd-bc2e-464cdaeed2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:20.067226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481b922-21f9-45cd-bc2e-464cdaeed2eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:20.067776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1592a-cd24-43b7-8537-cff4e44f8e5c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:20.068379 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481b922-21f9-45cd-bc2e-464cdaeed2eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:20.069382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 27/54] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:20.069872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481b922-21f9-45cd-bc2e-464cdaeed2eb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:20.071458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 28/55] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:20.075536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1592a-cd24-43b7-8537-cff4e44f8e5c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:20.077003 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481b922-21f9-45cd-bc2e-464cdaeed2eb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:20.122850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:20.175984 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1592a-cd24-43b7-8537-cff4e44f8e5c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:20.179146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481b922-21f9-45cd-bc2e-464cdaeed2eb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:20.208845 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591ada8-baaa-472a-94f6-b1a9fcb70b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:20.209457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ada8-baaa-472a-94f6-b1a9fcb70b27 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:20.209740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ada8-baaa-472a-94f6-b1a9fcb70b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:20.210010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ada8-baaa-472a-94f6-b1a9fcb70b27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:20.210733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591ada8-baaa-472a-94f6-b1a9fcb70b27 None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.domain.id=default, user.id=42a19c5d3ad549648071d18e7300d6c1) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:20.211038 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591ada8-baaa-472a-94f6-b1a9fcb70b27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:20.212541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591ada8-baaa-472a-94f6-b1a9fcb70b27 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:20.212952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139a702-bd61-4737-9804-fd9a4a7622e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:20.213848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139a702-bd61-4737-9804-fd9a4a7622e3 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:20.213848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139a702-bd61-4737-9804-fd9a4a7622e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:20.214134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139a702-bd61-4737-9804-fd9a4a7622e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:20.214795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6139a702-bd61-4737-9804-fd9a4a7622e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:20.216338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6139a702-bd61-4737-9804-fd9a4a7622e3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:20.221637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 28/56] 10.222.0.79 () {56 vars in 1322 bytes} [Tue Sep 6 20:59:20 2022] GET /identity/v3/role_assignments?user.id=42a19c5d3ad549648071d18e7300d6c1&scope.domain.id=default&role.id=3b30504991194002b41f8587d0851c10 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:20.246616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 29/57] 10.222.0.79 () {60 vars in 1140 bytes} [Tue Sep 6 20:59:20 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:59:22.387049 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b233dba-8700-468c-ad64-f8dd866cb4e5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:22.387627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b233dba-8700-468c-ad64-f8dd866cb4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:22.387710 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b233dba-8700-468c-ad64-f8dd866cb4e5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:22.389081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 29/58] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:22.394898 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149495f-85fb-424d-a35d-a1af9fa17678 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:22.395337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149495f-85fb-424d-a35d-a1af9fa17678 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:22.395418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149495f-85fb-424d-a35d-a1af9fa17678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:22.414048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b149495f-85fb-424d-a35d-a1af9fa17678 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:22.415241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b149495f-85fb-424d-a35d-a1af9fa17678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:22.426381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:22.485215 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 30/59] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:22.752691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b233dba-8700-468c-ad64-f8dd866cb4e5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:22.859012 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b233dba-8700-468c-ad64-f8dd866cb4e5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:22.869703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:22.895673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec18fd-9676-48cb-97b3-575e6b8eb38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:22.896292 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec18fd-9676-48cb-97b3-575e6b8eb38d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:22.896568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec18fd-9676-48cb-97b3-575e6b8eb38d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:22.896882 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec18fd-9676-48cb-97b3-575e6b8eb38d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:22.898051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec18fd-9676-48cb-97b3-575e6b8eb38d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:22.899525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec18fd-9676-48cb-97b3-575e6b8eb38d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:22.901129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 30/60] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:22.906909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149495f-85fb-424d-a35d-a1af9fa17678 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:23.006586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149495f-85fb-424d-a35d-a1af9fa17678 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:23.041195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1786bcb-9c68-4c98-8bba-ec10faf4fcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:23.041798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1786bcb-9c68-4c98-8bba-ec10faf4fcd3 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:23.042076 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1786bcb-9c68-4c98-8bba-ec10faf4fcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:23.042345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1786bcb-9c68-4c98-8bba-ec10faf4fcd3 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:23.042987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1786bcb-9c68-4c98-8bba-ec10faf4fcd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:23.044610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1786bcb-9c68-4c98-8bba-ec10faf4fcd3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:23.068492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:23.073379 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 31/61] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 20:59:22 2022] POST /identity/v3/users => generated 275 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:59:26.259901 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026bf98-7de9-4fc8-b66d-c2c4b4b97e7d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:26.261029 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026bf98-7de9-4fc8-b66d-c2c4b4b97e7d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:26.261509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026bf98-7de9-4fc8-b66d-c2c4b4b97e7d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:26.263386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 31/62] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:26.268469 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244cdc7-54b4-46d3-b828-ccca5cb7ac15 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:26.269305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244cdc7-54b4-46d3-b828-ccca5cb7ac15 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:26.269797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244cdc7-54b4-46d3-b828-ccca5cb7ac15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:26.293678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2244cdc7-54b4-46d3-b828-ccca5cb7ac15 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:26.295261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2244cdc7-54b4-46d3-b828-ccca5cb7ac15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:26.386383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 32/63] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:26.757849 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026bf98-7de9-4fc8-b66d-c2c4b4b97e7d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:26.921295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026bf98-7de9-4fc8-b66d-c2c4b4b97e7d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:26.974346 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cc599-2afa-4e0e-96c6-b05258b13ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:26.975009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cc599-2afa-4e0e-96c6-b05258b13ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:26.975284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cc599-2afa-4e0e-96c6-b05258b13ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:26.975548 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cc599-2afa-4e0e-96c6-b05258b13ef1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:26.991360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cc599-2afa-4e0e-96c6-b05258b13ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-047cc599-2afa-4e0e-96c6-b05258b13ef1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:26.996511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:26.997393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:26.998968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:26.998968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:26.998968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:26.998968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:26.998968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:26.998968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:26.998968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:26.998968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:26.998968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:26.998968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:26.998968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:26.998968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:26.998968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 32/64] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:59:27.005869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244cdc7-54b4-46d3-b828-ccca5cb7ac15 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:27.011068 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad20aa-cde6-4403-aa99-d2a01deaf790 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.011363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad20aa-cde6-4403-aa99-d2a01deaf790 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.011694 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad20aa-cde6-4403-aa99-d2a01deaf790 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.016292 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 33/65] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:59:27.018179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe61c7-ae49-42fa-b1a0-35f2456cc5f0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.018464 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe61c7-ae49-42fa-b1a0-35f2456cc5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.018735 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe61c7-ae49-42fa-b1a0-35f2456cc5f0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.020195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 34/66] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:59:27.021134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf02d1b-0470-4f41-8fbe-9f747a5a15ff None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.021414 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf02d1b-0470-4f41-8fbe-9f747a5a15ff None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.021685 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf02d1b-0470-4f41-8fbe-9f747a5a15ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.046076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fdf02d1b-0470-4f41-8fbe-9f747a5a15ff None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:27.047795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fdf02d1b-0470-4f41-8fbe-9f747a5a15ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:27.088708 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:27.125794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244cdc7-54b4-46d3-b828-ccca5cb7ac15 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:27.131750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 35/67] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:27.136938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c764a-1bdb-4e30-b2d8-57cc3d68d8b5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.136938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c764a-1bdb-4e30-b2d8-57cc3d68d8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.136938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c764a-1bdb-4e30-b2d8-57cc3d68d8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.156880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-104c764a-1bdb-4e30-b2d8-57cc3d68d8b5 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:27.158073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-104c764a-1bdb-4e30-b2d8-57cc3d68d8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:27.165262 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:27.174892 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f8a5-39ad-48fb-9e21-513f685e4098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:27.175450 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f8a5-39ad-48fb-9e21-513f685e4098 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.175737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f8a5-39ad-48fb-9e21-513f685e4098 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.176003 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f8a5-39ad-48fb-9e21-513f685e4098 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.176720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f8a5-39ad-48fb-9e21-513f685e4098 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:27.177027 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f8a5-39ad-48fb-9e21-513f685e4098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:27.178454 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f8a5-39ad-48fb-9e21-513f685e4098 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:27.187960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 33/68] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:27.194123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b9a49-0444-42c4-88fc-22afaaa9d7b5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.194718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b9a49-0444-42c4-88fc-22afaaa9d7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.194995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b9a49-0444-42c4-88fc-22afaaa9d7b5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.196767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 34/69] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:27.201665 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a797cc-870e-4b60-a153-450eb55c9722 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.201953 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a797cc-870e-4b60-a153-450eb55c9722 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.202225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a797cc-870e-4b60-a153-450eb55c9722 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.204165 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 35/70] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:27.208494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a797cc-870e-4b60-a153-450eb55c9722 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:27.245373 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 36/71] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:27.250283 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78b64b-a632-4ba9-8eb6-92643b52313a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.250739 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78b64b-a632-4ba9-8eb6-92643b52313a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.251010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78b64b-a632-4ba9-8eb6-92643b52313a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.270173 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5c78b64b-a632-4ba9-8eb6-92643b52313a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:27.271386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5c78b64b-a632-4ba9-8eb6-92643b52313a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:27.325427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a797cc-870e-4b60-a153-450eb55c9722 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:27.349102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 37/72] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:27.350512 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0681ad-cbdf-4c7f-af9b-9bcf8177100f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.350797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0681ad-cbdf-4c7f-af9b-9bcf8177100f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.351072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0681ad-cbdf-4c7f-af9b-9bcf8177100f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.366950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb0174-2f9a-473f-9af4-fc178c88683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:27.368596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb0174-2f9a-473f-9af4-fc178c88683a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.369118 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb0174-2f9a-473f-9af4-fc178c88683a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.371332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb0174-2f9a-473f-9af4-fc178c88683a None admin] PATH_INFO: `/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.390999 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0c0681ad-cbdf-4c7f-af9b-9bcf8177100f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:27.395213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0c0681ad-cbdf-4c7f-af9b-9bcf8177100f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:27.402385 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb0174-2f9a-473f-9af4-fc178c88683a None admin] RBAC: Authorizing `identity:get_user(user_id=40fbba8ecae846bca4e3b0fd6eaf8c85)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:27.404193 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb0174-2f9a-473f-9af4-fc178c88683a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:27.406807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 36/73] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:27 2022] GET /identity/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:27.407547 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb0174-2f9a-473f-9af4-fc178c88683a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:27.430108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:27.473150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:27.483682 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 38/74] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:27.489914 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0681ad-cbdf-4c7f-af9b-9bcf8177100f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:27.551969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb0174-2f9a-473f-9af4-fc178c88683a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:27.613746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0681ad-cbdf-4c7f-af9b-9bcf8177100f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:27.614420 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d18c10-a90c-47a3-9d5c-e6679bfff532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:27.615591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d18c10-a90c-47a3-9d5c-e6679bfff532 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.616982 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d18c10-a90c-47a3-9d5c-e6679bfff532 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.617160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d18c10-a90c-47a3-9d5c-e6679bfff532 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.630100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d18c10-a90c-47a3-9d5c-e6679bfff532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-03d18c10-a90c-47a3-9d5c-e6679bfff532 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:27.636581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:27.637485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:27.638247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:27.638247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:27.638247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:27.638247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:27.638247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:27.638247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:27.638247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:27.638247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:27.638247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:27.638247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:27.638247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 20:59:27.638247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:27.644837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 37/75] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 20:59:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:27.644837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d18c10-a90c-47a3-9d5c-e6679bfff532 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:27.660061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ddb40-a2c1-4772-abc9-084d7fd07d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:27.660716 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ddb40-a2c1-4772-abc9-084d7fd07d09 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.661011 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ddb40-a2c1-4772-abc9-084d7fd07d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.661284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ddb40-a2c1-4772-abc9-084d7fd07d09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.692148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:27.743612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ddb40-a2c1-4772-abc9-084d7fd07d09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:27.745263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ddb40-a2c1-4772-abc9-084d7fd07d09 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:27.754591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d18c10-a90c-47a3-9d5c-e6679bfff532 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:27.766957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:27.793465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fab6f-6de4-4d73-9de4-517268979a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:27.794083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fab6f-6de4-4d73-9de4-517268979a5e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.794372 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fab6f-6de4-4d73-9de4-517268979a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.794651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fab6f-6de4-4d73-9de4-517268979a5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.805421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483fab6f-6de4-4d73-9de4-517268979a5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-483fab6f-6de4-4d73-9de4-517268979a5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:27.810440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:27.811812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:27.812573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:27.812573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:27.812573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:27.812573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:27.812573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:27.812573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:27.812573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:27.812573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:27.812573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:27.812573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:27.812573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:27.812573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:27.812573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 38/76] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:27.813239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fab6f-6de4-4d73-9de4-517268979a5e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:27.852871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 39/77] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:27.853585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ddb40-a2c1-4772-abc9-084d7fd07d09 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:27.928525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:27.940935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fab6f-6de4-4d73-9de4-517268979a5e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:27.973976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ddb40-a2c1-4772-abc9-084d7fd07d09 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:27.976369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d93dd-1c4d-48c2-840b-2c6b98c866e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:27.977062 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d93dd-1c4d-48c2-840b-2c6b98c866e5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:27.977345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d93dd-1c4d-48c2-840b-2c6b98c866e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:27.977619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d93dd-1c4d-48c2-840b-2c6b98c866e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:27.988955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d93dd-1c4d-48c2-840b-2c6b98c866e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-884d93dd-1c4d-48c2-840b-2c6b98c866e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:27.993841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:27.994723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:27.995551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:27.995551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:27.995551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:27.995551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:27.995551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:27.995551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:27.995551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:27.995551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:27.995551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:27.995551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:27.995551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:27.995551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:27.995551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 39/78] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 20:59:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:27.996149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d93dd-1c4d-48c2-840b-2c6b98c866e5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:28.011285 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd864b0-3c78-4caa-8633-173811929adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:28.011285 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd864b0-3c78-4caa-8633-173811929adf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:28.011285 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd864b0-3c78-4caa-8633-173811929adf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:28.011285 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd864b0-3c78-4caa-8633-173811929adf None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:28.012137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd864b0-3c78-4caa-8633-173811929adf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:28.012137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd864b0-3c78-4caa-8633-173811929adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:28.014443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd864b0-3c78-4caa-8633-173811929adf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:28.023354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 40/79] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 20:59:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:28.024283 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd864b0-3c78-4caa-8633-173811929adf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:28.034055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:28.107990 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d93dd-1c4d-48c2-840b-2c6b98c866e5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:28.128394 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd864b0-3c78-4caa-8633-173811929adf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:28.142435 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d853af7-5c40-4559-a201-7768c06aeb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:28.143077 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d853af7-5c40-4559-a201-7768c06aeb10 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:28.143360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d853af7-5c40-4559-a201-7768c06aeb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:28.143633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d853af7-5c40-4559-a201-7768c06aeb10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:28.144778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d853af7-5c40-4559-a201-7768c06aeb10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:28.146257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d853af7-5c40-4559-a201-7768c06aeb10 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:28.147833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 40/80] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:28.148868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d853af7-5c40-4559-a201-7768c06aeb10 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:28.149954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:28.164232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c342f-6379-4557-ac4a-6236679a8d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:28.164877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c342f-6379-4557-ac4a-6236679a8d98 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:28.165160 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c342f-6379-4557-ac4a-6236679a8d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:28.165434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c342f-6379-4557-ac4a-6236679a8d98 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:28.166102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c342f-6379-4557-ac4a-6236679a8d98 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:28.166545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c342f-6379-4557-ac4a-6236679a8d98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:28.167961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c342f-6379-4557-ac4a-6236679a8d98 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:28.176410 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 41/81] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:28.177065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c342f-6379-4557-ac4a-6236679a8d98 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:28.265335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:28.269409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d853af7-5c40-4559-a201-7768c06aeb10 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:28.300396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c342f-6379-4557-ac4a-6236679a8d98 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:28.307751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f5162-36af-4d5b-9297-27fa9a971cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:28.308573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f5162-36af-4d5b-9297-27fa9a971cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:28.309077 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f5162-36af-4d5b-9297-27fa9a971cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:28.309411 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f5162-36af-4d5b-9297-27fa9a971cf6 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:28.310831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f5162-36af-4d5b-9297-27fa9a971cf6 None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:28.312930 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f5162-36af-4d5b-9297-27fa9a971cf6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:28.314889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 41/82] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:28 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:28.315774 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f5162-36af-4d5b-9297-27fa9a971cf6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:28.335792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11848ad-8af0-40c5-8046-303f6dbb103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:28.336400 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11848ad-8af0-40c5-8046-303f6dbb103d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:28.336696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11848ad-8af0-40c5-8046-303f6dbb103d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:28.336957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11848ad-8af0-40c5-8046-303f6dbb103d None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:28.337623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11848ad-8af0-40c5-8046-303f6dbb103d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:28.337920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11848ad-8af0-40c5-8046-303f6dbb103d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:28.339335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11848ad-8af0-40c5-8046-303f6dbb103d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:28.347853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 42/83] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:28.348625 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11848ad-8af0-40c5-8046-303f6dbb103d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:28.402749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:28.429552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f5162-36af-4d5b-9297-27fa9a971cf6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:28.460889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11848ad-8af0-40c5-8046-303f6dbb103d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:28.463712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141ebba-48a5-4c53-9e49-39b97ef5756b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:28.464475 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141ebba-48a5-4c53-9e49-39b97ef5756b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:28.464779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141ebba-48a5-4c53-9e49-39b97ef5756b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:28.465057 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141ebba-48a5-4c53-9e49-39b97ef5756b None admin] PATH_INFO: `/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:28.466203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141ebba-48a5-4c53-9e49-39b97ef5756b None admin] RBAC: Authorizing `identity:get_user(user_id=40fbba8ecae846bca4e3b0fd6eaf8c85)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:28.467776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141ebba-48a5-4c53-9e49-39b97ef5756b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:28.469426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 42/84] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:28 2022] GET /identity/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:28.470737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141ebba-48a5-4c53-9e49-39b97ef5756b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:28.511068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:28.513813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94595ef-ebd1-443c-98aa-241e75b1a35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:28.514520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94595ef-ebd1-443c-98aa-241e75b1a35d None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:28.514786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94595ef-ebd1-443c-98aa-241e75b1a35d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:28.514992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94595ef-ebd1-443c-98aa-241e75b1a35d None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:28.515693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94595ef-ebd1-443c-98aa-241e75b1a35d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:28.517124 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94595ef-ebd1-443c-98aa-241e75b1a35d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:28.545325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 43/85] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 20:59:28 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:59:28.546053 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94595ef-ebd1-443c-98aa-241e75b1a35d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:28.597199 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141ebba-48a5-4c53-9e49-39b97ef5756b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:28.637333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7381908-0334-45a5-b9cb-50d700c273fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:28.637990 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7381908-0334-45a5-b9cb-50d700c273fe None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:28.638268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7381908-0334-45a5-b9cb-50d700c273fe None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:28.638579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7381908-0334-45a5-b9cb-50d700c273fe None admin] PATH_INFO: `/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:28.639942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7381908-0334-45a5-b9cb-50d700c273fe None admin] RBAC: Authorizing `identity:get_user(user_id=40fbba8ecae846bca4e3b0fd6eaf8c85)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:28.641953 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7381908-0334-45a5-b9cb-50d700c273fe None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:28.643847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 43/86] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:28 2022] GET /identity/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:28.644700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7381908-0334-45a5-b9cb-50d700c273fe None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:28.674156 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:28.678293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94595ef-ebd1-443c-98aa-241e75b1a35d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:28.731297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142731b-7b5a-417a-ade6-56701bf7a38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:28.731961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142731b-7b5a-417a-ade6-56701bf7a38e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:28.732244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142731b-7b5a-417a-ade6-56701bf7a38e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:28.732614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142731b-7b5a-417a-ade6-56701bf7a38e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:28.733533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142731b-7b5a-417a-ade6-56701bf7a38e None admin] RBAC: Adding query filter params (role.id=180edbb8a9e74555af89d6a7639f819a, scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e, user.id=40fbba8ecae846bca4e3b0fd6eaf8c85) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:28.733842 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142731b-7b5a-417a-ade6-56701bf7a38e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:28.735341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142731b-7b5a-417a-ade6-56701bf7a38e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:28.743365 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 44/87] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:28 2022] GET /identity/v3/role_assignments?user.id=40fbba8ecae846bca4e3b0fd6eaf8c85&scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e&role.id=180edbb8a9e74555af89d6a7639f819a => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:28.744222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142731b-7b5a-417a-ade6-56701bf7a38e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:28.790187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7381908-0334-45a5-b9cb-50d700c273fe None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:28.792201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:28.841413 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb125240-6772-47fe-b944-96f2e5ec2a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:28.842081 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb125240-6772-47fe-b944-96f2e5ec2a56 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:28.842351 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb125240-6772-47fe-b944-96f2e5ec2a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:28.842826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb125240-6772-47fe-b944-96f2e5ec2a56 None admin] PATH_INFO: `/v3/users/42a19c5d3ad549648071d18e7300d6c1` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:28.847216 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb125240-6772-47fe-b944-96f2e5ec2a56 None admin] RBAC: Authorizing `identity:get_user(user_id=42a19c5d3ad549648071d18e7300d6c1)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:28.847216 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb125240-6772-47fe-b944-96f2e5ec2a56 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:28.848553 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 44/88] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:28 2022] GET /identity/v3/users/42a19c5d3ad549648071d18e7300d6c1 => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:28.856849 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb125240-6772-47fe-b944-96f2e5ec2a56 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:28.916079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142731b-7b5a-417a-ade6-56701bf7a38e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:28.959876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fea9c-bb6b-4b3d-b52b-04ee88f139f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:28.960891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fea9c-bb6b-4b3d-b52b-04ee88f139f7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:28.961377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fea9c-bb6b-4b3d-b52b-04ee88f139f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:28.961837 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fea9c-bb6b-4b3d-b52b-04ee88f139f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:28.980279 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb125240-6772-47fe-b944-96f2e5ec2a56 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:28.999074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:29.031837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e60944-1b4d-4649-a62c-c546ca029783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:29.032947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e60944-1b4d-4649-a62c-c546ca029783 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:29.033232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e60944-1b4d-4649-a62c-c546ca029783 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:29.033535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e60944-1b4d-4649-a62c-c546ca029783 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:29.049437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857fea9c-bb6b-4b3d-b52b-04ee88f139f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:29.051055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857fea9c-bb6b-4b3d-b52b-04ee88f139f7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:29.089125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:29.124556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e60944-1b4d-4649-a62c-c546ca029783 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:29.127332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e60944-1b4d-4649-a62c-c546ca029783 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:29.178509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 45/89] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:29.179526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fea9c-bb6b-4b3d-b52b-04ee88f139f7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:29.250318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 45/90] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:29.251628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e60944-1b4d-4649-a62c-c546ca029783 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:29.335649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:29.344240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:29.379156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e60944-1b4d-4649-a62c-c546ca029783 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:29.419342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdff73-6ba2-47da-982b-3e1b2ae778a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:29.420394 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdff73-6ba2-47da-982b-3e1b2ae778a1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:29.420694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdff73-6ba2-47da-982b-3e1b2ae778a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:29.420973 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdff73-6ba2-47da-982b-3e1b2ae778a1 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:29.422148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdff73-6ba2-47da-982b-3e1b2ae778a1 None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:29.423790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdff73-6ba2-47da-982b-3e1b2ae778a1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:29.425491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 46/91] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:29 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:29.426412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdff73-6ba2-47da-982b-3e1b2ae778a1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:29.453557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fea9c-bb6b-4b3d-b52b-04ee88f139f7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:29.543259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdff73-6ba2-47da-982b-3e1b2ae778a1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:29.568048 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e896412-b78a-4287-a5ef-40c268f1658f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:29.569567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e896412-b78a-4287-a5ef-40c268f1658f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:29.570060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e896412-b78a-4287-a5ef-40c268f1658f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:29.570812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e896412-b78a-4287-a5ef-40c268f1658f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:29.581505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f1255-050c-4fa0-a5de-0de1386a55db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:29.582163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f1255-050c-4fa0-a5de-0de1386a55db None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:29.582442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f1255-050c-4fa0-a5de-0de1386a55db None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:29.582717 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f1255-050c-4fa0-a5de-0de1386a55db None admin] PATH_INFO: `/v3/projects/576e6cec81664d5b903fbd4439ab041b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:29.583898 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f1255-050c-4fa0-a5de-0de1386a55db None admin] RBAC: Authorizing `identity:get_project(project_id=576e6cec81664d5b903fbd4439ab041b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:29.588101 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f1255-050c-4fa0-a5de-0de1386a55db None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:29.588101 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 47/92] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:29 2022] GET /identity/v3/projects/576e6cec81664d5b903fbd4439ab041b => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:29.588920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f1255-050c-4fa0-a5de-0de1386a55db None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:29.601314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:29.721438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e896412-b78a-4287-a5ef-40c268f1658f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:29.722943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e896412-b78a-4287-a5ef-40c268f1658f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:29.788128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f1255-050c-4fa0-a5de-0de1386a55db None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:29.847280 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:29.865391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473798b-2324-4f5a-84b2-76d2deb987b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:29.866023 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473798b-2324-4f5a-84b2-76d2deb987b0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:29.866300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473798b-2324-4f5a-84b2-76d2deb987b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:29.866572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473798b-2324-4f5a-84b2-76d2deb987b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:29.868753 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473798b-2324-4f5a-84b2-76d2deb987b0 None admin] RBAC: Adding query filter params (role.id=e945f087a5bc4c38bd8848782f76c214, scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e, user.id=40fbba8ecae846bca4e3b0fd6eaf8c85) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:29.869070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473798b-2324-4f5a-84b2-76d2deb987b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:29.870566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473798b-2324-4f5a-84b2-76d2deb987b0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:29.887270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 48/93] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:29 2022] GET /identity/v3/role_assignments?user.id=40fbba8ecae846bca4e3b0fd6eaf8c85&scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e&role.id=e945f087a5bc4c38bd8848782f76c214 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:29.888469 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473798b-2324-4f5a-84b2-76d2deb987b0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:29.922742 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 46/94] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 744 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:29.928686 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e896412-b78a-4287-a5ef-40c268f1658f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:30.019202 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:30.038135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e896412-b78a-4287-a5ef-40c268f1658f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:30.079563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473798b-2324-4f5a-84b2-76d2deb987b0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:30.084353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cba6b-9e19-4fcd-8f41-81d5c52ede45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:30.085276 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cba6b-9e19-4fcd-8f41-81d5c52ede45 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:30.085746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cba6b-9e19-4fcd-8f41-81d5c52ede45 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:30.086172 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cba6b-9e19-4fcd-8f41-81d5c52ede45 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:30.087717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cba6b-9e19-4fcd-8f41-81d5c52ede45 None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:30.089529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cba6b-9e19-4fcd-8f41-81d5c52ede45 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:30.100947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 47/95] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:29 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:30.106854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cba6b-9e19-4fcd-8f41-81d5c52ede45 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:30.167251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b0073-4e36-48ed-8862-5e41b75a7241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:30.168546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b0073-4e36-48ed-8862-5e41b75a7241 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:30.169032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:30.169523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b0073-4e36-48ed-8862-5e41b75a7241 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:30.170081 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b0073-4e36-48ed-8862-5e41b75a7241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:30.171224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b0073-4e36-48ed-8862-5e41b75a7241 None admin] RBAC: Adding query filter params (role.id=180edbb8a9e74555af89d6a7639f819a, scope.project.id=576e6cec81664d5b903fbd4439ab041b, user.id=40fbba8ecae846bca4e3b0fd6eaf8c85) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:30.172058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b0073-4e36-48ed-8862-5e41b75a7241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:30.173798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b0073-4e36-48ed-8862-5e41b75a7241 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:30.191834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 49/96] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:29 2022] GET /identity/v3/role_assignments?user.id=40fbba8ecae846bca4e3b0fd6eaf8c85&scope.project.id=576e6cec81664d5b903fbd4439ab041b&role.id=180edbb8a9e74555af89d6a7639f819a => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:30.286066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cba6b-9e19-4fcd-8f41-81d5c52ede45 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:30.342009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0d566-332c-4835-abab-983085b3c14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:30.342643 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0d566-332c-4835-abab-983085b3c14a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:30.342907 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0d566-332c-4835-abab-983085b3c14a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:30.343178 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0d566-332c-4835-abab-983085b3c14a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:30.343926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0d566-332c-4835-abab-983085b3c14a None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e, user.id=42a19c5d3ad549648071d18e7300d6c1) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:30.344300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0d566-332c-4835-abab-983085b3c14a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:30.345907 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0d566-332c-4835-abab-983085b3c14a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:30.357820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 48/97] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:30 2022] GET /identity/v3/role_assignments?user.id=42a19c5d3ad549648071d18e7300d6c1&scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e&role.id=3b30504991194002b41f8587d0851c10 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:31.435204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044344c-a79c-4618-82b8-9064b6db8133 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:31.435819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044344c-a79c-4618-82b8-9064b6db8133 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:31.436032 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044344c-a79c-4618-82b8-9064b6db8133 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:31.438299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 50/98] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:31.443948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9945653-729f-4f8c-9f3b-f7926bbdb6d4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:31.444824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9945653-729f-4f8c-9f3b-f7926bbdb6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:31.445297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9945653-729f-4f8c-9f3b-f7926bbdb6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:31.469674 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a9945653-729f-4f8c-9f3b-f7926bbdb6d4 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:31.471332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a9945653-729f-4f8c-9f3b-f7926bbdb6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:31.558607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 49/99] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:31.818909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a849e8-0576-43ce-bc21-01d2c0bccfcd None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:31.819599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a849e8-0576-43ce-bc21-01d2c0bccfcd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:31.820078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a849e8-0576-43ce-bc21-01d2c0bccfcd None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:31.821676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 51/100] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:31.834311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff500aa-07d6-4485-b84b-4522798ac4b1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:31.834905 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff500aa-07d6-4485-b84b-4522798ac4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:31.835372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff500aa-07d6-4485-b84b-4522798ac4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:31.859923 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bff500aa-07d6-4485-b84b-4522798ac4b1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:31.861500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bff500aa-07d6-4485-b84b-4522798ac4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:31.891964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:31.903146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a849e8-0576-43ce-bc21-01d2c0bccfcd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:31.976445 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 50/101] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:32.084620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a849e8-0576-43ce-bc21-01d2c0bccfcd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:32.126512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dec62-012e-4060-8cc3-ab53069af9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:32.127612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dec62-012e-4060-8cc3-ab53069af9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:32.128138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dec62-012e-4060-8cc3-ab53069af9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:32.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dec62-012e-4060-8cc3-ab53069af9e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:32.133895 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:32.151877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357dec62-012e-4060-8cc3-ab53069af9e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-357dec62-012e-4060-8cc3-ab53069af9e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:32.157145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:32.158065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:32.158834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:32.158834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:32.158834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:32.158834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:32.158834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:32.158834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:32.158834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:32.158834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:32.158834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:32.158834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:32.158834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:32.158834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:32.158834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 52/102] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:59:32.164471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff500aa-07d6-4485-b84b-4522798ac4b1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:32.310860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896053d8-ab61-4902-a99b-5b54a2fc47c9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:32.311179 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896053d8-ab61-4902-a99b-5b54a2fc47c9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:32.311804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896053d8-ab61-4902-a99b-5b54a2fc47c9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:32.313487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 53/103] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:32.318976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c559409-e9c4-4fa3-bd3e-c7b2601919d6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:32.319263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c559409-e9c4-4fa3-bd3e-c7b2601919d6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:32.319541 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c559409-e9c4-4fa3-bd3e-c7b2601919d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:32.322743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff500aa-07d6-4485-b84b-4522798ac4b1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:32.355325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3c559409-e9c4-4fa3-bd3e-c7b2601919d6 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:32.356605 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3c559409-e9c4-4fa3-bd3e-c7b2601919d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:32.374389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed4b20-e7e5-4bd7-a13c-3b5ee222b9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:32.375027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed4b20-e7e5-4bd7-a13c-3b5ee222b9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:32.375309 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed4b20-e7e5-4bd7-a13c-3b5ee222b9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:32.375586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed4b20-e7e5-4bd7-a13c-3b5ee222b9d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:32.376270 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed4b20-e7e5-4bd7-a13c-3b5ee222b9d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:32.376575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed4b20-e7e5-4bd7-a13c-3b5ee222b9d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:32.378005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed4b20-e7e5-4bd7-a13c-3b5ee222b9d8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:32.391114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 51/104] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:32.392318 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed4b20-e7e5-4bd7-a13c-3b5ee222b9d8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:32.440319 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:32.496439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 54/105] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:32.497387 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c559409-e9c4-4fa3-bd3e-c7b2601919d6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:32.606735 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed4b20-e7e5-4bd7-a13c-3b5ee222b9d8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:32.692015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2540a0-426b-4413-aa1c-9c2bba81448e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:32.692743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2540a0-426b-4413-aa1c-9c2bba81448e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:32.693074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2540a0-426b-4413-aa1c-9c2bba81448e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:32.693361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2540a0-426b-4413-aa1c-9c2bba81448e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:32.697429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2540a0-426b-4413-aa1c-9c2bba81448e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:32.699090 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2540a0-426b-4413-aa1c-9c2bba81448e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:32.701103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 52/106] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:32.709626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2540a0-426b-4413-aa1c-9c2bba81448e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:32.712473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c559409-e9c4-4fa3-bd3e-c7b2601919d6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:32.753530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:32.769225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3cb92-d970-4a71-8769-f1ae72c6b9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:32.769891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3cb92-d970-4a71-8769-f1ae72c6b9c8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:32.770174 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3cb92-d970-4a71-8769-f1ae72c6b9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:32.770452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3cb92-d970-4a71-8769-f1ae72c6b9c8 None admin] PATH_INFO: `/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:32.771616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3cb92-d970-4a71-8769-f1ae72c6b9c8 None admin] RBAC: Authorizing `identity:get_user(user_id=40fbba8ecae846bca4e3b0fd6eaf8c85)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:32.773265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3cb92-d970-4a71-8769-f1ae72c6b9c8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:32.775007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 55/107] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:32 2022] GET /identity/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85 => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:32.785210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3cb92-d970-4a71-8769-f1ae72c6b9c8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:32.921973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:32.923552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3cb92-d970-4a71-8769-f1ae72c6b9c8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:32.961731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6e2cc-d8e7-4a60-bf9c-69900a8f0259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:32.962396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6e2cc-d8e7-4a60-bf9c-69900a8f0259 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:32.962670 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6e2cc-d8e7-4a60-bf9c-69900a8f0259 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:32.962879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2540a0-426b-4413-aa1c-9c2bba81448e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:32.963009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6e2cc-d8e7-4a60-bf9c-69900a8f0259 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:32.964151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6e2cc-d8e7-4a60-bf9c-69900a8f0259 None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:32.965716 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6e2cc-d8e7-4a60-bf9c-69900a8f0259 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:32.967807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 56/108] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:32 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:32.968609 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6e2cc-d8e7-4a60-bf9c-69900a8f0259 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:33.046759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:33.053453 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee4e79-939f-40c1-adee-89ebe77159e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:33.054148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee4e79-939f-40c1-adee-89ebe77159e5 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:33.054421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee4e79-939f-40c1-adee-89ebe77159e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:33.054689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee4e79-939f-40c1-adee-89ebe77159e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:33.055460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee4e79-939f-40c1-adee-89ebe77159e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:33.057015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee4e79-939f-40c1-adee-89ebe77159e5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:33.071540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6e2cc-d8e7-4a60-bf9c-69900a8f0259 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:33.110233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26094964-d61d-4d55-95a7-706b75f187d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:33.110938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26094964-d61d-4d55-95a7-706b75f187d4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:33.111274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26094964-d61d-4d55-95a7-706b75f187d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:33.111862 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26094964-d61d-4d55-95a7-706b75f187d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:33.121469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 53/109] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 20:59:32 2022] POST /identity/v3/projects => generated 302 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:59:33.122177 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee4e79-939f-40c1-adee-89ebe77159e5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:33.124449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26094964-d61d-4d55-95a7-706b75f187d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-26094964-d61d-4d55-95a7-706b75f187d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:33.130150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:33.131110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:33.131864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:33.131864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:33.131864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:33.131864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:33.131864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:33.131864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:33.131864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:33.131864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:33.131864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:33.131864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:33.131864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 20:59:33.131864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:33.131864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 57/110] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:33.142560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26094964-d61d-4d55-95a7-706b75f187d4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:33.358858 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26094964-d61d-4d55-95a7-706b75f187d4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:33.363436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee4e79-939f-40c1-adee-89ebe77159e5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:33.399377 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:33.427835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f0f84-6c5e-4012-89c6-c75525119ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:33.428648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f0f84-6c5e-4012-89c6-c75525119ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:33.429024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f0f84-6c5e-4012-89c6-c75525119ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:33.429304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f0f84-6c5e-4012-89c6-c75525119ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:33.429995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f0f84-6c5e-4012-89c6-c75525119ce1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:33.430301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f0f84-6c5e-4012-89c6-c75525119ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:33.431750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f0f84-6c5e-4012-89c6-c75525119ce1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:33.433499 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58992962-4480-4f1c-a354-6d945e54f22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:33.434299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58992962-4480-4f1c-a354-6d945e54f22a None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:33.434643 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58992962-4480-4f1c-a354-6d945e54f22a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:33.435000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58992962-4480-4f1c-a354-6d945e54f22a None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e/users/40fbba8ecae846bca4e3b0fd6eaf8c85/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:33.438441 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58992962-4480-4f1c-a354-6d945e54f22a None admin] RBAC: Authorizing `identity:create_grant(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e, user_id=40fbba8ecae846bca4e3b0fd6eaf8c85, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:33.440023 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58992962-4480-4f1c-a354-6d945e54f22a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:33.444817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 58/111] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:33.446214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e20532-058d-4570-81b0-620328c1cfc2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:33.446605 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e20532-058d-4570-81b0-620328c1cfc2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:33.446893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e20532-058d-4570-81b0-620328c1cfc2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:33.448611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 59/112] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:33.452478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e20532-058d-4570-81b0-620328c1cfc2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:33.455715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:33.464335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 54/113] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 20:59:33 2022] PUT /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e/users/40fbba8ecae846bca4e3b0fd6eaf8c85/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:59:33.466965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d0a18-703b-455a-b98b-4e2638b372bc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:33.467250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d0a18-703b-455a-b98b-4e2638b372bc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:33.467562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d0a18-703b-455a-b98b-4e2638b372bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:33.491669 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e95d0a18-703b-455a-b98b-4e2638b372bc None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:33.494099 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e95d0a18-703b-455a-b98b-4e2638b372bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:33.624383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e20532-058d-4570-81b0-620328c1cfc2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:33.633271 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 55/114] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:33.634653 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3467e91-e17d-42c6-97ce-78d55518cce4 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:33.634935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3467e91-e17d-42c6-97ce-78d55518cce4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:33.635259 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3467e91-e17d-42c6-97ce-78d55518cce4 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:33.637063 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 56/115] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:33.644891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1b53f-8183-45e0-a8ae-3ed785541699 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:33.645204 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1b53f-8183-45e0-a8ae-3ed785541699 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:33.645442 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1b53f-8183-45e0-a8ae-3ed785541699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:33.665515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e443f-625a-4cbf-9d1c-93274fca4640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:33.667427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e443f-625a-4cbf-9d1c-93274fca4640 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:33.667714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e443f-625a-4cbf-9d1c-93274fca4640 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:33.667991 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e443f-625a-4cbf-9d1c-93274fca4640 None admin] PATH_INFO: `/v3/users/72a9afd5d53e4c1a97376bda65fbdd64` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:33.684036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e443f-625a-4cbf-9d1c-93274fca4640 None admin] RBAC: Authorizing `identity:get_user(user_id=72a9afd5d53e4c1a97376bda65fbdd64)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:33.685912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e443f-625a-4cbf-9d1c-93274fca4640 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:33.688081 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 60/116] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:33 2022] GET /identity/v3/users/72a9afd5d53e4c1a97376bda65fbdd64 => generated 289 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:33.688220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f9b1b53f-8183-45e0-a8ae-3ed785541699 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:33.689450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f9b1b53f-8183-45e0-a8ae-3ed785541699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:33.694779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e443f-625a-4cbf-9d1c-93274fca4640 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:33.760716 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:33.818095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 57/117] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:33.819477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82b2be-d7bc-4660-9744-f30a522e818a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:33.819673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82b2be-d7bc-4660-9744-f30a522e818a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:33.819903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82b2be-d7bc-4660-9744-f30a522e818a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:33.821349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 58/118] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:33.826946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670648a-e739-4d6f-affb-0c9cec9a01e0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:33.827225 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670648a-e739-4d6f-affb-0c9cec9a01e0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:33.827506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670648a-e739-4d6f-affb-0c9cec9a01e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:33.828899 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e443f-625a-4cbf-9d1c-93274fca4640 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:33.850184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6670648a-e739-4d6f-affb-0c9cec9a01e0 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:33.851923 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6670648a-e739-4d6f-affb-0c9cec9a01e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:33.867282 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f8d75-3dc3-47f5-9cdd-0a60da4066dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:33.868043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f8d75-3dc3-47f5-9cdd-0a60da4066dc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:33.868324 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f8d75-3dc3-47f5-9cdd-0a60da4066dc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:33.868669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f8d75-3dc3-47f5-9cdd-0a60da4066dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:33.912107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:33.926823 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 59/119] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:33.956099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f8d75-3dc3-47f5-9cdd-0a60da4066dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:33.957623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f8d75-3dc3-47f5-9cdd-0a60da4066dc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:34.012913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:34.090333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 61/120] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 20:59:34.096214 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670648a-e739-4d6f-affb-0c9cec9a01e0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:34.101044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f8d75-3dc3-47f5-9cdd-0a60da4066dc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:34.213458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f8d75-3dc3-47f5-9cdd-0a60da4066dc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:34.250541 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91550f-bf29-4ee0-b00d-fa54988f8aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:34.250541 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91550f-bf29-4ee0-b00d-fa54988f8aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:34.250541 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91550f-bf29-4ee0-b00d-fa54988f8aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:34.250541 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91550f-bf29-4ee0-b00d-fa54988f8aa8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:34.258499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:34.262816 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91550f-bf29-4ee0-b00d-fa54988f8aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ac91550f-bf29-4ee0-b00d-fa54988f8aa8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:34.267938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:34.268847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:34.269611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:34.269611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:34.269611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:34.269611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:34.269611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:34.269611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:34.269611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:34.269611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:34.269611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:34.269611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:34.269611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 20:59:34.269611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:34.274524 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 62/121] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 20:59:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:59:34.280843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91550f-bf29-4ee0-b00d-fa54988f8aa8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:34.282317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670648a-e739-4d6f-affb-0c9cec9a01e0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:34.320147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f042a47-1a2e-4b34-b988-6f88a081e945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:34.321289 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f042a47-1a2e-4b34-b988-6f88a081e945 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:34.321805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f042a47-1a2e-4b34-b988-6f88a081e945 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:34.322261 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f042a47-1a2e-4b34-b988-6f88a081e945 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:34.323801 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f042a47-1a2e-4b34-b988-6f88a081e945 None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:34.325601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f042a47-1a2e-4b34-b988-6f88a081e945 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:34.327979 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 60/122] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:34 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:34.335030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f042a47-1a2e-4b34-b988-6f88a081e945 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:34.421438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91550f-bf29-4ee0-b00d-fa54988f8aa8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:34.435893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:34.470939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ec62b-453d-4182-8c0a-b7a07810bac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:34.471500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ec62b-453d-4182-8c0a-b7a07810bac7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:34.471806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ec62b-453d-4182-8c0a-b7a07810bac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:34.472096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ec62b-453d-4182-8c0a-b7a07810bac7 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:34.472891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ec62b-453d-4182-8c0a-b7a07810bac7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:34.473508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ec62b-453d-4182-8c0a-b7a07810bac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:34.474719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ec62b-453d-4182-8c0a-b7a07810bac7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:34.484190 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 63/123] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 20:59:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:34.489224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ec62b-453d-4182-8c0a-b7a07810bac7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:34.504082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f042a47-1a2e-4b34-b988-6f88a081e945 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:34.544098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbdcbc-78de-483c-bd0b-80db0b432ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:34.545126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbdcbc-78de-483c-bd0b-80db0b432ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:34.546127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbdcbc-78de-483c-bd0b-80db0b432ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:34.546614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbdcbc-78de-483c-bd0b-80db0b432ed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:34.547540 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbdcbc-78de-483c-bd0b-80db0b432ed7 None admin] RBAC: Adding query filter params (role.id=6abadafbec7f43d2b2cc2df439c44eff, scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e, user.id=72a9afd5d53e4c1a97376bda65fbdd64) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:34.548488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbdcbc-78de-483c-bd0b-80db0b432ed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:34.550822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbdcbc-78de-483c-bd0b-80db0b432ed7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:34.565306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 61/124] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:34 2022] GET /identity/v3/role_assignments?user.id=72a9afd5d53e4c1a97376bda65fbdd64&scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e&role.id=6abadafbec7f43d2b2cc2df439c44eff => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:34.565306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbdcbc-78de-483c-bd0b-80db0b432ed7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:34.593880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:34.714020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbdcbc-78de-483c-bd0b-80db0b432ed7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:34.714939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ec62b-453d-4182-8c0a-b7a07810bac7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:34.741641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:34.769753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f6a10-0ce7-4f21-b022-2f03f482712b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:34.770867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f6a10-0ce7-4f21-b022-2f03f482712b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:34.771379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f6a10-0ce7-4f21-b022-2f03f482712b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:34.771868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f6a10-0ce7-4f21-b022-2f03f482712b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:34.787350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ceb5d-634d-4c64-99ad-0cd7c8d7eb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:34.788040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f6a10-0ce7-4f21-b022-2f03f482712b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:34.788180 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ceb5d-634d-4c64-99ad-0cd7c8d7eb73 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:34.788370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ceb5d-634d-4c64-99ad-0cd7c8d7eb73 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:34.788594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ceb5d-634d-4c64-99ad-0cd7c8d7eb73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-df3f6a10-0ce7-4f21-b022-2f03f482712b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:34.794804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:34.795770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:34.796511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:34.796511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:34.796511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:34.796511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:34.796511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:34.796511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:34.796511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:34.796511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:34.796511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:34.796511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:34.796511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:34.796511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:34.799396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 62/125] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 20:59:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:34.800719 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f6a10-0ce7-4f21-b022-2f03f482712b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:34.818144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ceb5d-634d-4c64-99ad-0cd7c8d7eb73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-528ceb5d-634d-4c64-99ad-0cd7c8d7eb73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:34.824737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:34.825671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:34.826508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:34.826508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:34.826508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:34.826508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:34.826508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:34.826508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:34.826508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:34.826508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:34.826508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:34.826508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:34.826508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:34.826508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:34.826508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 64/126] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:34.827139 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ceb5d-634d-4c64-99ad-0cd7c8d7eb73 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:34.958803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f6a10-0ce7-4f21-b022-2f03f482712b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:34.995052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f68d7-29c4-4e7f-9e58-9d33ecadb536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:34.996053 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f68d7-29c4-4e7f-9e58-9d33ecadb536 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:34.996525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f68d7-29c4-4e7f-9e58-9d33ecadb536 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:34.997013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f68d7-29c4-4e7f-9e58-9d33ecadb536 None admin] PATH_INFO: `/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:34.998434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f68d7-29c4-4e7f-9e58-9d33ecadb536 None admin] RBAC: Authorizing `identity:get_user(user_id=40fbba8ecae846bca4e3b0fd6eaf8c85)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:35.000090 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f68d7-29c4-4e7f-9e58-9d33ecadb536 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:35.001977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 63/127] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:34 2022] GET /identity/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:35.003026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f68d7-29c4-4e7f-9e58-9d33ecadb536 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:35.054468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:35.088149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ceb5d-634d-4c64-99ad-0cd7c8d7eb73 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:35.137015 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:35.150398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f68d7-29c4-4e7f-9e58-9d33ecadb536 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:35.158706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431131d-5ecb-4e41-9c21-c70b56d60bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:35.159340 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431131d-5ecb-4e41-9c21-c70b56d60bfb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:35.159622 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431131d-5ecb-4e41-9c21-c70b56d60bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:35.159897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431131d-5ecb-4e41-9c21-c70b56d60bfb None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:35.160578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0431131d-5ecb-4e41-9c21-c70b56d60bfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:35.160906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0431131d-5ecb-4e41-9c21-c70b56d60bfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:35.162311 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0431131d-5ecb-4e41-9c21-c70b56d60bfb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:35.173288 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 65/128] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:35.174407 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431131d-5ecb-4e41-9c21-c70b56d60bfb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:35.209951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae15b3c-f56c-4cb6-a244-a11a103884ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:35.211042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae15b3c-f56c-4cb6-a244-a11a103884ca None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:35.211513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae15b3c-f56c-4cb6-a244-a11a103884ca None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:35.211980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae15b3c-f56c-4cb6-a244-a11a103884ca None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:35.212974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae15b3c-f56c-4cb6-a244-a11a103884ca None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:35.213467 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae15b3c-f56c-4cb6-a244-a11a103884ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:35.215296 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae15b3c-f56c-4cb6-a244-a11a103884ca None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:35.224836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 64/129] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:35.226037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae15b3c-f56c-4cb6-a244-a11a103884ca None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:35.292599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431131d-5ecb-4e41-9c21-c70b56d60bfb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:35.330967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b4323-676b-4ae0-8304-59ff4648b236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:35.331631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b4323-676b-4ae0-8304-59ff4648b236 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:35.331912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b4323-676b-4ae0-8304-59ff4648b236 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:35.332183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b4323-676b-4ae0-8304-59ff4648b236 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:35.333356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b4323-676b-4ae0-8304-59ff4648b236 None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:35.334839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b4323-676b-4ae0-8304-59ff4648b236 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:35.336611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 66/130] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:35 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:35.337397 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b4323-676b-4ae0-8304-59ff4648b236 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:35.395905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:35.477805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b4323-676b-4ae0-8304-59ff4648b236 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:35.503807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae15b3c-f56c-4cb6-a244-a11a103884ca None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:35.523532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:35.545630 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb929c90-0b4e-413b-8921-6ff4c35fc95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:35.546276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb929c90-0b4e-413b-8921-6ff4c35fc95f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:35.546560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb929c90-0b4e-413b-8921-6ff4c35fc95f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:35.546837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb929c90-0b4e-413b-8921-6ff4c35fc95f None admin] PATH_INFO: `/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:35.553429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb929c90-0b4e-413b-8921-6ff4c35fc95f None admin] RBAC: Authorizing `identity:get_user(user_id=40fbba8ecae846bca4e3b0fd6eaf8c85)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:35.554919 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb929c90-0b4e-413b-8921-6ff4c35fc95f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:35.556772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 67/131] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:35 2022] GET /identity/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:35.557472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb929c90-0b4e-413b-8921-6ff4c35fc95f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:35.560618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960e2c7-0290-485f-b53a-9c87ef68f19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:35.561656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960e2c7-0290-485f-b53a-9c87ef68f19f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:35.561937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960e2c7-0290-485f-b53a-9c87ef68f19f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:35.562210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960e2c7-0290-485f-b53a-9c87ef68f19f None admin] PATH_INFO: `/v3/users/42a19c5d3ad549648071d18e7300d6c1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:35.573712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960e2c7-0290-485f-b53a-9c87ef68f19f None admin] RBAC: Authorizing `identity:get_user(user_id=42a19c5d3ad549648071d18e7300d6c1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:35.575250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960e2c7-0290-485f-b53a-9c87ef68f19f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:35.577117 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 65/132] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:35 2022] GET /identity/v3/users/42a19c5d3ad549648071d18e7300d6c1 => generated 247 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:35.578053 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960e2c7-0290-485f-b53a-9c87ef68f19f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:35.683700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb929c90-0b4e-413b-8921-6ff4c35fc95f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:35.709509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:35.713980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960e2c7-0290-485f-b53a-9c87ef68f19f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:35.723337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c502a-2f66-4e9d-b4da-3f7402a06be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:35.724602 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c502a-2f66-4e9d-b4da-3f7402a06be3 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:35.725090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c502a-2f66-4e9d-b4da-3f7402a06be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:35.725562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c502a-2f66-4e9d-b4da-3f7402a06be3 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e/users/40fbba8ecae846bca4e3b0fd6eaf8c85/roles/e945f087a5bc4c38bd8848782f76c214` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:35.727933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c502a-2f66-4e9d-b4da-3f7402a06be3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e, user_id=40fbba8ecae846bca4e3b0fd6eaf8c85, role_id=e945f087a5bc4c38bd8848782f76c214)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:35.729739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c502a-2f66-4e9d-b4da-3f7402a06be3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:35.747244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 68/133] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 20:59:35 2022] PUT /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e/users/40fbba8ecae846bca4e3b0fd6eaf8c85/roles/e945f087a5bc4c38bd8848782f76c214 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:59:35.748531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c502a-2f66-4e9d-b4da-3f7402a06be3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:35.788292 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf16d2-b940-439c-9e8f-d3af7240632d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:35.789122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf16d2-b940-439c-9e8f-d3af7240632d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:35.789481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf16d2-b940-439c-9e8f-d3af7240632d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:35.789829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf16d2-b940-439c-9e8f-d3af7240632d None admin] PATH_INFO: `/v3/projects/576e6cec81664d5b903fbd4439ab041b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:35.792995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf16d2-b940-439c-9e8f-d3af7240632d None admin] RBAC: Authorizing `identity:get_project(project_id=576e6cec81664d5b903fbd4439ab041b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:35.794708 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf16d2-b940-439c-9e8f-d3af7240632d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:35.797659 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 66/134] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:35 2022] GET /identity/v3/projects/576e6cec81664d5b903fbd4439ab041b => generated 312 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:35.805874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf16d2-b940-439c-9e8f-d3af7240632d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:35.868571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:35.889517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c502a-2f66-4e9d-b4da-3f7402a06be3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:35.937557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ef55a-9127-4bf9-8257-55e1397b0dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:35.939000 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ef55a-9127-4bf9-8257-55e1397b0dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:35.939423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ef55a-9127-4bf9-8257-55e1397b0dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:35.939988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ef55a-9127-4bf9-8257-55e1397b0dc3 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:35.941466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ef55a-9127-4bf9-8257-55e1397b0dc3 None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:35.943474 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ef55a-9127-4bf9-8257-55e1397b0dc3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:35.946303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 69/135] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:35 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:35.954480 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ef55a-9127-4bf9-8257-55e1397b0dc3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:35.995377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf16d2-b940-439c-9e8f-d3af7240632d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:36.030159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:36.069396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db10b5a-31de-41db-b299-ebd2552d207c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:36.069603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db10b5a-31de-41db-b299-ebd2552d207c None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:36.069826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db10b5a-31de-41db-b299-ebd2552d207c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:36.070098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db10b5a-31de-41db-b299-ebd2552d207c None admin] PATH_INFO: `/v3/projects/576e6cec81664d5b903fbd4439ab041b/users/40fbba8ecae846bca4e3b0fd6eaf8c85/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:36.072395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db10b5a-31de-41db-b299-ebd2552d207c None admin] RBAC: Authorizing `identity:create_grant(project_id=576e6cec81664d5b903fbd4439ab041b, user_id=40fbba8ecae846bca4e3b0fd6eaf8c85, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:36.074094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db10b5a-31de-41db-b299-ebd2552d207c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:36.091799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 67/136] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 20:59:35 2022] PUT /identity/v3/projects/576e6cec81664d5b903fbd4439ab041b/users/40fbba8ecae846bca4e3b0fd6eaf8c85/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:59:36.093488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafea5ed-cba9-4b0f-bb18-d84b3ddf6565 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:36.093998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafea5ed-cba9-4b0f-bb18-d84b3ddf6565 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:36.094300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafea5ed-cba9-4b0f-bb18-d84b3ddf6565 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:36.096153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 68/137] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:36.102119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6f542-b3bd-4219-a7b1-193b5a0a2dfc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:36.102496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6f542-b3bd-4219-a7b1-193b5a0a2dfc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:36.102837 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6f542-b3bd-4219-a7b1-193b5a0a2dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:36.103138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ef55a-9127-4bf9-8257-55e1397b0dc3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:36.124441 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3cf6f542-b3bd-4219-a7b1-193b5a0a2dfc None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:36.127896 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3cf6f542-b3bd-4219-a7b1-193b5a0a2dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:36.147711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55dd79-b348-4a29-8aeb-f091676d399a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:36.148846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55dd79-b348-4a29-8aeb-f091676d399a None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:36.149320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55dd79-b348-4a29-8aeb-f091676d399a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:36.149791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55dd79-b348-4a29-8aeb-f091676d399a None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e/users/42a19c5d3ad549648071d18e7300d6c1/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:36.152308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55dd79-b348-4a29-8aeb-f091676d399a None admin] RBAC: Authorizing `identity:create_grant(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e, user_id=42a19c5d3ad549648071d18e7300d6c1, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:36.154234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55dd79-b348-4a29-8aeb-f091676d399a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:36.174848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 70/138] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 20:59:35 2022] PUT /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e/users/42a19c5d3ad549648071d18e7300d6c1/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:59:36.226146 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 69/139] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:36.610158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55dd79-b348-4a29-8aeb-f091676d399a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:36.673276 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906972c-79f5-4ea7-80b3-363b51f82ee3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:36.673867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906972c-79f5-4ea7-80b3-363b51f82ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:36.674319 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906972c-79f5-4ea7-80b3-363b51f82ee3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:36.676121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 70/140] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:36.681574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861ad62-65ce-4050-9277-699fdba6c853 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:36.682073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861ad62-65ce-4050-9277-699fdba6c853 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:36.682526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861ad62-65ce-4050-9277-699fdba6c853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:36.713671 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6861ad62-65ce-4050-9277-699fdba6c853 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:36.714872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6861ad62-65ce-4050-9277-699fdba6c853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:36.739657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55dd79-b348-4a29-8aeb-f091676d399a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:36.752589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:36.778286 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64270d-1d84-4fc1-9334-4d132a2c6db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:36.779315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64270d-1d84-4fc1-9334-4d132a2c6db0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:36.779806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64270d-1d84-4fc1-9334-4d132a2c6db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:36.780309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64270d-1d84-4fc1-9334-4d132a2c6db0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:36.781776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64270d-1d84-4fc1-9334-4d132a2c6db0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:36.783694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64270d-1d84-4fc1-9334-4d132a2c6db0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:36.785986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 71/141] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:36.792112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64270d-1d84-4fc1-9334-4d132a2c6db0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:36.794998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:36.884164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 71/142] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:36.943571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64270d-1d84-4fc1-9334-4d132a2c6db0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:36.993343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1a6ef-6abc-4697-9adf-ed043ad5f4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:36.993972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1a6ef-6abc-4697-9adf-ed043ad5f4ae None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:36.994255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1a6ef-6abc-4697-9adf-ed043ad5f4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:36.994653 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1a6ef-6abc-4697-9adf-ed043ad5f4ae None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:37.001276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1a6ef-6abc-4697-9adf-ed043ad5f4ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:37.002795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1a6ef-6abc-4697-9adf-ed043ad5f4ae None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:37.027708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 72/143] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 20:59:36 2022] POST /identity/v3/users => generated 283 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:59:37.255599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861ad62-65ce-4050-9277-699fdba6c853 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:37.385552 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861ad62-65ce-4050-9277-699fdba6c853 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:37.432303 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7340a1-2f06-46e5-a61e-6be6021e9fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:37.433333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7340a1-2f06-46e5-a61e-6be6021e9fbc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:37.433824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7340a1-2f06-46e5-a61e-6be6021e9fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:37.434319 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7340a1-2f06-46e5-a61e-6be6021e9fbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:37.450188 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7340a1-2f06-46e5-a61e-6be6021e9fbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-af7340a1-2f06-46e5-a61e-6be6021e9fbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:37.455511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:37.456399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:37.457205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:37.457205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:37.457205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:37.457205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:37.457205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:37.457205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:37.457205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:37.457205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:37.457205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:37.457205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:37.457205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:37.457205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:37.459564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 72/144] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:37.466753 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1a6ef-6abc-4697-9adf-ed043ad5f4ae None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:37.519131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:37.643056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1a6ef-6abc-4697-9adf-ed043ad5f4ae None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:37.703212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb46add-553c-406c-aa4c-ade999c9a5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:37.703859 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb46add-553c-406c-aa4c-ade999c9a5ab None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:37.704144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb46add-553c-406c-aa4c-ade999c9a5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:37.704416 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb46add-553c-406c-aa4c-ade999c9a5ab None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:37.708111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb46add-553c-406c-aa4c-ade999c9a5ab None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:37.708111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb46add-553c-406c-aa4c-ade999c9a5ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:37.708305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975cda0-eee8-40da-afa9-1df7ccebe91a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:37.708305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975cda0-eee8-40da-afa9-1df7ccebe91a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:37.708305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975cda0-eee8-40da-afa9-1df7ccebe91a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:37.708509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb46add-553c-406c-aa4c-ade999c9a5ab None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:37.708581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 73/145] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:37.716423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 73/146] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:37.718274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d378be-5107-4c60-bb79-d8e05940cedc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:37.718790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d378be-5107-4c60-bb79-d8e05940cedc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:37.719260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d378be-5107-4c60-bb79-d8e05940cedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:37.722034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb46add-553c-406c-aa4c-ade999c9a5ab None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:37.752730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e9d378be-5107-4c60-bb79-d8e05940cedc None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:37.754307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e9d378be-5107-4c60-bb79-d8e05940cedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:37.806020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:37.857097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:37.866756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb46add-553c-406c-aa4c-ade999c9a5ab None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:37.890871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 74/147] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:37.912921 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdcfd0-34e3-4e08-bd64-2e139de71b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:37.913967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdcfd0-34e3-4e08-bd64-2e139de71b5f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:37.914722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdcfd0-34e3-4e08-bd64-2e139de71b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:37.915145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdcfd0-34e3-4e08-bd64-2e139de71b5f None admin] PATH_INFO: `/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:37.916481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bdcfd0-34e3-4e08-bd64-2e139de71b5f None admin] RBAC: Authorizing `identity:get_user(user_id=40fbba8ecae846bca4e3b0fd6eaf8c85)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:37.918262 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bdcfd0-34e3-4e08-bd64-2e139de71b5f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:37.920209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 74/148] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:37 2022] GET /identity/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:37.929838 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d378be-5107-4c60-bb79-d8e05940cedc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:38.051300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d378be-5107-4c60-bb79-d8e05940cedc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:38.092172 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2b54f-73a7-405a-b300-25cc59f77898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:38.093124 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2b54f-73a7-405a-b300-25cc59f77898 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:38.093596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2b54f-73a7-405a-b300-25cc59f77898 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:38.094042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2b54f-73a7-405a-b300-25cc59f77898 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:38.123036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:38.175753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2b54f-73a7-405a-b300-25cc59f77898 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:38.177676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2b54f-73a7-405a-b300-25cc59f77898 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:38.291980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 75/149] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:38.297984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdcfd0-34e3-4e08-bd64-2e139de71b5f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:38.341392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2b54f-73a7-405a-b300-25cc59f77898 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:38.423079 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdcfd0-34e3-4e08-bd64-2e139de71b5f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:38.451197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:38.467353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255bab0-7942-4169-a156-3b092bab27d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:38.468372 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255bab0-7942-4169-a156-3b092bab27d9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:38.468867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255bab0-7942-4169-a156-3b092bab27d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:38.469362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255bab0-7942-4169-a156-3b092bab27d9 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:38.470921 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255bab0-7942-4169-a156-3b092bab27d9 None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:38.473321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255bab0-7942-4169-a156-3b092bab27d9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:38.475688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 75/150] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:38 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:38.482587 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255bab0-7942-4169-a156-3b092bab27d9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:38.518803 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:38.541186 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2b54f-73a7-405a-b300-25cc59f77898 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:38.598236 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255bab0-7942-4169-a156-3b092bab27d9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:38.623856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1e4ff-79ec-4afd-95b8-67f1b77ee695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:38.624503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1e4ff-79ec-4afd-95b8-67f1b77ee695 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:38.624947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1e4ff-79ec-4afd-95b8-67f1b77ee695 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:38.625214 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1e4ff-79ec-4afd-95b8-67f1b77ee695 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:38.638192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1e4ff-79ec-4afd-95b8-67f1b77ee695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e1a1e4ff-79ec-4afd-95b8-67f1b77ee695 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:38.644793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:38.645660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:38.646409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:38.646409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:38.646409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:38.646409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:38.646409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:38.646409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:38.646409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:38.646409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:38.646409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:38.646409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:38.646409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 20:59:38.646409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:38.646409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 76/151] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:38.647817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4f006-b749-45ff-86c6-53e8ba13e4a8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:38.648456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4f006-b749-45ff-86c6-53e8ba13e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:38.648456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4f006-b749-45ff-86c6-53e8ba13e4a8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:38.650430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 77/152] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:38.653727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3911c2a-138d-4cb6-84a2-4bb0ea04a04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:38.654877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3911c2a-138d-4cb6-84a2-4bb0ea04a04f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:38.655348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3911c2a-138d-4cb6-84a2-4bb0ea04a04f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:38.655837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3911c2a-138d-4cb6-84a2-4bb0ea04a04f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:38.656072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4f006-b749-45ff-86c6-53e8ba13e4a8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:38.657711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3911c2a-138d-4cb6-84a2-4bb0ea04a04f None admin] RBAC: Adding query filter params (role.id=180edbb8a9e74555af89d6a7639f819a, scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e, user.id=40fbba8ecae846bca4e3b0fd6eaf8c85) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:38.658180 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3911c2a-138d-4cb6-84a2-4bb0ea04a04f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:38.660131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3911c2a-138d-4cb6-84a2-4bb0ea04a04f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:38.668542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 76/153] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:38 2022] GET /identity/v3/role_assignments?user.id=40fbba8ecae846bca4e3b0fd6eaf8c85&scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e&role.id=180edbb8a9e74555af89d6a7639f819a => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:38.670065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b849071-a3a9-432f-8b2e-9cef99187e0d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:38.670353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b849071-a3a9-432f-8b2e-9cef99187e0d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:38.670628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b849071-a3a9-432f-8b2e-9cef99187e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:38.713438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4b849071-a3a9-432f-8b2e-9cef99187e0d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:38.714836 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4b849071-a3a9-432f-8b2e-9cef99187e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:38.830950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 77/154] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:38.868183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4f006-b749-45ff-86c6-53e8ba13e4a8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:38.889968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:38.912356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501918db-f74e-4ada-84e3-bb72a03db2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:38.913024 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501918db-f74e-4ada-84e3-bb72a03db2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:38.913363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501918db-f74e-4ada-84e3-bb72a03db2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:38.913639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501918db-f74e-4ada-84e3-bb72a03db2f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:38.915298 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501918db-f74e-4ada-84e3-bb72a03db2f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:38.915608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501918db-f74e-4ada-84e3-bb72a03db2f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:38.917583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501918db-f74e-4ada-84e3-bb72a03db2f1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:38.933935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 78/155] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:38.940770 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b849071-a3a9-432f-8b2e-9cef99187e0d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:38.983007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:39.074766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b849071-a3a9-432f-8b2e-9cef99187e0d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:39.111292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d4ff0-b684-489a-9f33-adc07517f1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:39.111948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d4ff0-b684-489a-9f33-adc07517f1fa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:39.112221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d4ff0-b684-489a-9f33-adc07517f1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:39.112486 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d4ff0-b684-489a-9f33-adc07517f1fa None admin] PATH_INFO: `/v3/users/72a9afd5d53e4c1a97376bda65fbdd64` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:39.113658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d4ff0-b684-489a-9f33-adc07517f1fa None admin] RBAC: Authorizing `identity:get_user(user_id=72a9afd5d53e4c1a97376bda65fbdd64)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:39.115255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d4ff0-b684-489a-9f33-adc07517f1fa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:39.117292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 78/156] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:38 2022] GET /identity/v3/users/72a9afd5d53e4c1a97376bda65fbdd64 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:39.123334 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501918db-f74e-4ada-84e3-bb72a03db2f1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:39.269168 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501918db-f74e-4ada-84e3-bb72a03db2f1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:39.306600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e72cd-10b1-4849-b781-a459df400e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:39.307846 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e72cd-10b1-4849-b781-a459df400e7d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:39.308336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e72cd-10b1-4849-b781-a459df400e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:39.308894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e72cd-10b1-4849-b781-a459df400e7d None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:39.310675 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e72cd-10b1-4849-b781-a459df400e7d None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:39.312790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e72cd-10b1-4849-b781-a459df400e7d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:39.314779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 79/157] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:39 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:39.320734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d4ff0-b684-489a-9f33-adc07517f1fa None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:39.341334 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e72cd-10b1-4849-b781-a459df400e7d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:39.419923 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:39.469808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:39.484615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d4ff0-b684-489a-9f33-adc07517f1fa None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:39.493159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e72cd-10b1-4849-b781-a459df400e7d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:39.538965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fa5e3-ad59-4438-809d-2a280ebedefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:39.539975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fa5e3-ad59-4438-809d-2a280ebedefd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:39.540457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fa5e3-ad59-4438-809d-2a280ebedefd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:39.541243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fa5e3-ad59-4438-809d-2a280ebedefd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:39.566098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc71839-ffd2-4f5e-9ed2-26a74feda582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:39.566792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc71839-ffd2-4f5e-9ed2-26a74feda582 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:39.567062 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc71839-ffd2-4f5e-9ed2-26a74feda582 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:39.567368 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc71839-ffd2-4f5e-9ed2-26a74feda582 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e/users/72a9afd5d53e4c1a97376bda65fbdd64/roles/6abadafbec7f43d2b2cc2df439c44eff` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:39.570233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc71839-ffd2-4f5e-9ed2-26a74feda582 None admin] RBAC: Authorizing `identity:create_grant(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e, user_id=72a9afd5d53e4c1a97376bda65fbdd64, role_id=6abadafbec7f43d2b2cc2df439c44eff)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:39.571345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fa5e3-ad59-4438-809d-2a280ebedefd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:39.572534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc71839-ffd2-4f5e-9ed2-26a74feda582 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-918fa5e3-ad59-4438-809d-2a280ebedefd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:39.578736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:39.579900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:39.581237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:39.581237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:39.581237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:39.581237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:39.581237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:39.581237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:39.581237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:39.581237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:39.581237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:39.581237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:39.581237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 20:59:39.581237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:39.581237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 80/158] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 20:59:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:59:39.586900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fa5e3-ad59-4438-809d-2a280ebedefd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:39.601716 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 79/159] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 20:59:39 2022] PUT /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e/users/72a9afd5d53e4c1a97376bda65fbdd64/roles/6abadafbec7f43d2b2cc2df439c44eff => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:59:39.735402 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fa5e3-ad59-4438-809d-2a280ebedefd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:39.743600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:39.791068 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3f34a-a523-4f46-bdf1-fb4204a5a9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:39.791616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3f34a-a523-4f46-bdf1-fb4204a5a9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:39.791948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3f34a-a523-4f46-bdf1-fb4204a5a9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:39.792221 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3f34a-a523-4f46-bdf1-fb4204a5a9b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:39.793137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3f34a-a523-4f46-bdf1-fb4204a5a9b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:39.793443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3f34a-a523-4f46-bdf1-fb4204a5a9b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:39.794978 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3f34a-a523-4f46-bdf1-fb4204a5a9b7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:39.806161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 81/160] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 20:59:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:39.812979 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc71839-ffd2-4f5e-9ed2-26a74feda582 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:39.942587 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc71839-ffd2-4f5e-9ed2-26a74feda582 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:39.982759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4df50d-501c-4e93-aa70-f6716631d438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:39.983431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4df50d-501c-4e93-aa70-f6716631d438 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:39.983707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4df50d-501c-4e93-aa70-f6716631d438 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:39.983975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4df50d-501c-4e93-aa70-f6716631d438 None admin] PATH_INFO: `/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:39.985262 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4df50d-501c-4e93-aa70-f6716631d438 None admin] RBAC: Authorizing `identity:get_user(user_id=40fbba8ecae846bca4e3b0fd6eaf8c85)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:39.986899 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4df50d-501c-4e93-aa70-f6716631d438 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:39.988604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 80/161] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:39 2022] GET /identity/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:39.994650 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3f34a-a523-4f46-bdf1-fb4204a5a9b7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:40.120313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3f34a-a523-4f46-bdf1-fb4204a5a9b7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:40.155142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1050c6-edef-4ffc-96da-38fb4aee511d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:40.155768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1050c6-edef-4ffc-96da-38fb4aee511d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:40.156045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1050c6-edef-4ffc-96da-38fb4aee511d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:40.156314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1050c6-edef-4ffc-96da-38fb4aee511d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:40.201132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:40.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1050c6-edef-4ffc-96da-38fb4aee511d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:40.245866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1050c6-edef-4ffc-96da-38fb4aee511d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:40.366131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 82/162] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:40.372358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4df50d-501c-4e93-aa70-f6716631d438 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:40.419377 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:40.527322 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4df50d-501c-4e93-aa70-f6716631d438 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:40.562408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33a6e6-5f42-45e0-af69-fb5cccb3652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:40.563115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33a6e6-5f42-45e0-af69-fb5cccb3652e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:40.563399 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33a6e6-5f42-45e0-af69-fb5cccb3652e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:40.563669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33a6e6-5f42-45e0-af69-fb5cccb3652e None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:40.564860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33a6e6-5f42-45e0-af69-fb5cccb3652e None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:40.566345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33a6e6-5f42-45e0-af69-fb5cccb3652e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:40.572624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 81/163] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:40 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:40.574475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1050c6-edef-4ffc-96da-38fb4aee511d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:40.652315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32922779-fad8-4014-968a-3494b6e22c63 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:40.652612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32922779-fad8-4014-968a-3494b6e22c63 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:40.652832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32922779-fad8-4014-968a-3494b6e22c63 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:40.654518 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 82/164] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:40.660835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b180a4c-f43e-4fe2-b539-e2d197b4334f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:40.661114 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b180a4c-f43e-4fe2-b539-e2d197b4334f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:40.661389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b180a4c-f43e-4fe2-b539-e2d197b4334f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:40.689615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7b180a4c-f43e-4fe2-b539-e2d197b4334f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:40.690839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7b180a4c-f43e-4fe2-b539-e2d197b4334f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:40.693132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:40.742711 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1050c6-edef-4ffc-96da-38fb4aee511d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:40.791105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86141e0d-e4f9-4787-a6d9-ae044200d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:40.791727 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86141e0d-e4f9-4787-a6d9-ae044200d1be None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:40.792001 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86141e0d-e4f9-4787-a6d9-ae044200d1be None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:40.792266 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86141e0d-e4f9-4787-a6d9-ae044200d1be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:40.793264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86141e0d-e4f9-4787-a6d9-ae044200d1be None admin] RBAC: Adding query filter params (role.id=e945f087a5bc4c38bd8848782f76c214, scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e, user.id=40fbba8ecae846bca4e3b0fd6eaf8c85) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:40.793820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86141e0d-e4f9-4787-a6d9-ae044200d1be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:40.795739 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86141e0d-e4f9-4787-a6d9-ae044200d1be None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:40.802497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 83/165] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:40 2022] GET /identity/v3/role_assignments?user.id=40fbba8ecae846bca4e3b0fd6eaf8c85&scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e&role.id=e945f087a5bc4c38bd8848782f76c214 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:40.803979 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac2764-72e6-4797-8241-0ac9c0dcf0b5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:40.804260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac2764-72e6-4797-8241-0ac9c0dcf0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:40.805037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac2764-72e6-4797-8241-0ac9c0dcf0b5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:40.806507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 84/166] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:40.812524 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd11ff5-e607-4e45-b166-2ac4ebad7c1b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:40.813112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd11ff5-e607-4e45-b166-2ac4ebad7c1b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:40.813383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd11ff5-e607-4e45-b166-2ac4ebad7c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:40.843268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 83/167] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:40.849531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0fd11ff5-e607-4e45-b166-2ac4ebad7c1b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:40.850793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0fd11ff5-e607-4e45-b166-2ac4ebad7c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:40.967217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 85/168] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:41.254109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b180a4c-f43e-4fe2-b539-e2d197b4334f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:41.278480 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:41.393524 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b180a4c-f43e-4fe2-b539-e2d197b4334f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:41.432469 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1d80d-04f6-4055-87e2-6ed2d6eb27b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:41.433328 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1d80d-04f6-4055-87e2-6ed2d6eb27b9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:41.433488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1d80d-04f6-4055-87e2-6ed2d6eb27b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:41.433732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1d80d-04f6-4055-87e2-6ed2d6eb27b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:41.451013 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1d80d-04f6-4055-87e2-6ed2d6eb27b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-65a1d80d-04f6-4055-87e2-6ed2d6eb27b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:41.456031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:41.457373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:41.458658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:41.458658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:41.458658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:41.458658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:41.458658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:41.458658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:41.458658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:41.458658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:41.458658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:41.458658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:41.458658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:41.458658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:41.458658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 84/169] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 20:59:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:41.464718 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd11ff5-e607-4e45-b166-2ac4ebad7c1b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:41.558225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1d80d-04f6-4055-87e2-6ed2d6eb27b9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:41.573887 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:41.659428 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd11ff5-e607-4e45-b166-2ac4ebad7c1b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:41.724859 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:41.731829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5ec42-2e5b-4f71-9979-196a6f8d664e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:41.732553 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5ec42-2e5b-4f71-9979-196a6f8d664e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:41.732859 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5ec42-2e5b-4f71-9979-196a6f8d664e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:41.733133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5ec42-2e5b-4f71-9979-196a6f8d664e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:41.733813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5ec42-2e5b-4f71-9979-196a6f8d664e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:41.734116 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5ec42-2e5b-4f71-9979-196a6f8d664e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:41.735547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5ec42-2e5b-4f71-9979-196a6f8d664e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:41.751589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1d80d-04f6-4055-87e2-6ed2d6eb27b9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:41.759219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 86/170] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:41.764576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5ec42-2e5b-4f71-9979-196a6f8d664e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:41.795050 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1488f5-4283-4bdc-b997-e02e367522dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:41.796078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1488f5-4283-4bdc-b997-e02e367522dc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:41.796596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1488f5-4283-4bdc-b997-e02e367522dc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:41.797282 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1488f5-4283-4bdc-b997-e02e367522dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:41.812471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1488f5-4283-4bdc-b997-e02e367522dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0f1488f5-4283-4bdc-b997-e02e367522dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:41.819016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:41.819992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:41.820891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:41.820891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:41.820891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:41.820891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:41.820891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:41.820891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:41.820891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:41.820891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:41.820891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:41.820891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:41.820891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:41.820891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:41.823215 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 85/171] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:41.830388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1488f5-4283-4bdc-b997-e02e367522dc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:42.011717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5ec42-2e5b-4f71-9979-196a6f8d664e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:42.026280 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1488f5-4283-4bdc-b997-e02e367522dc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:42.056639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:42.076940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538e6f7-4608-41c6-b1b6-bf9dec3e0579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:42.077504 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538e6f7-4608-41c6-b1b6-bf9dec3e0579 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:42.077778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538e6f7-4608-41c6-b1b6-bf9dec3e0579 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:42.078043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538e6f7-4608-41c6-b1b6-bf9dec3e0579 None admin] PATH_INFO: `/v3/users/42a19c5d3ad549648071d18e7300d6c1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:42.079946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538e6f7-4608-41c6-b1b6-bf9dec3e0579 None admin] RBAC: Authorizing `identity:get_user(user_id=42a19c5d3ad549648071d18e7300d6c1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:42.082947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538e6f7-4608-41c6-b1b6-bf9dec3e0579 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:42.089884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 87/172] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:41 2022] GET /identity/v3/users/42a19c5d3ad549648071d18e7300d6c1 => generated 247 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:42.091994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874f4b2-a8b0-42ce-afb2-08b2f2e242f8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:42.092273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874f4b2-a8b0-42ce-afb2-08b2f2e242f8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:42.092543 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874f4b2-a8b0-42ce-afb2-08b2f2e242f8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:42.094099 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 88/173] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:42.098580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e651370-ed31-4cbc-9545-95bb9fbd95ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:42.099026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e651370-ed31-4cbc-9545-95bb9fbd95ea None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:42.099299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e651370-ed31-4cbc-9545-95bb9fbd95ea None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:42.099566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e651370-ed31-4cbc-9545-95bb9fbd95ea None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:42.100246 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e651370-ed31-4cbc-9545-95bb9fbd95ea None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:42.100542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e651370-ed31-4cbc-9545-95bb9fbd95ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:42.101683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874f4b2-a8b0-42ce-afb2-08b2f2e242f8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:42.103021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e651370-ed31-4cbc-9545-95bb9fbd95ea None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:42.113130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:42.122742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 86/174] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:42.124036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82486a0f-9bb9-4878-bef5-57c2e64e03cf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:42.124319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82486a0f-9bb9-4878-bef5-57c2e64e03cf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:42.124599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82486a0f-9bb9-4878-bef5-57c2e64e03cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:42.163187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-82486a0f-9bb9-4878-bef5-57c2e64e03cf None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:42.165899 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-82486a0f-9bb9-4878-bef5-57c2e64e03cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:42.294697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 87/175] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:42.295444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82486a0f-9bb9-4878-bef5-57c2e64e03cf None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:42.297307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874f4b2-a8b0-42ce-afb2-08b2f2e242f8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:42.363032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88deb21b-f4ec-42ab-8d08-77c545c35d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:42.363584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb21b-f4ec-42ab-8d08-77c545c35d2d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:42.363866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb21b-f4ec-42ab-8d08-77c545c35d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:42.364136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb21b-f4ec-42ab-8d08-77c545c35d2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:42.477984 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:42.479935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82486a0f-9bb9-4878-bef5-57c2e64e03cf None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:42.500228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88deb21b-f4ec-42ab-8d08-77c545c35d2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:42.502129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88deb21b-f4ec-42ab-8d08-77c545c35d2d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:42.518907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:42.527540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be235d0-ce93-4286-932f-66b5af7e59fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:42.528817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be235d0-ce93-4286-932f-66b5af7e59fa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:42.529271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be235d0-ce93-4286-932f-66b5af7e59fa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:42.529908 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be235d0-ce93-4286-932f-66b5af7e59fa None admin] PATH_INFO: `/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:42.532157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be235d0-ce93-4286-932f-66b5af7e59fa None admin] RBAC: Authorizing `identity:get_user(user_id=40fbba8ecae846bca4e3b0fd6eaf8c85)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:42.534676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be235d0-ce93-4286-932f-66b5af7e59fa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:42.537312 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 88/176] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:42 2022] GET /identity/v3/users/40fbba8ecae846bca4e3b0fd6eaf8c85 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:42.538953 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cfd36-3d84-4855-af3d-5c6bb7e90760 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:42.539404 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cfd36-3d84-4855-af3d-5c6bb7e90760 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:42.539836 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cfd36-3d84-4855-af3d-5c6bb7e90760 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:42.546082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 89/177] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:42.549552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cfd36-3d84-4855-af3d-5c6bb7e90760 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:42.681690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cfd36-3d84-4855-af3d-5c6bb7e90760 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:42.720997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f7872-d887-47f1-b703-d2a926437b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:42.722006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f7872-d887-47f1-b703-d2a926437b50 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:42.722490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f7872-d887-47f1-b703-d2a926437b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:42.722951 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f7872-d887-47f1-b703-d2a926437b50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:42.739153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 89/178] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:42.740537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936835e-9d6b-4070-a102-6f912e1731eb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:42.744773 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936835e-9d6b-4070-a102-6f912e1731eb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:42.744773 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936835e-9d6b-4070-a102-6f912e1731eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:42.762739 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f936835e-9d6b-4070-a102-6f912e1731eb None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:42.763986 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f936835e-9d6b-4070-a102-6f912e1731eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:42.803471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:42.811493 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f7872-d887-47f1-b703-d2a926437b50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:42.813696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f7872-d887-47f1-b703-d2a926437b50 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:42.837620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 90/179] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:42.848827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936835e-9d6b-4070-a102-6f912e1731eb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:42.874902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:42.921018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 90/180] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:42.926624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f7872-d887-47f1-b703-d2a926437b50 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:42.977370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936835e-9d6b-4070-a102-6f912e1731eb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:43.017505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a8381-1a4a-4548-8bac-11cf2761ad49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:43.018382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a8381-1a4a-4548-8bac-11cf2761ad49 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.018757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a8381-1a4a-4548-8bac-11cf2761ad49 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.019180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a8381-1a4a-4548-8bac-11cf2761ad49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.032626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a8381-1a4a-4548-8bac-11cf2761ad49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-fc9a8381-1a4a-4548-8bac-11cf2761ad49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:43.037865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:43.038834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:43.039618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:43.039618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:43.039618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:43.039618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:43.039618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:43.039618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:43.039618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:43.039618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:43.039618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:43.039618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:43.039618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:43.039618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:43.039618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 91/181] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 20:59:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:43.045609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283a524-687a-4435-a70e-f9df62bf5bb5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.045609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283a524-687a-4435-a70e-f9df62bf5bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.045609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283a524-687a-4435-a70e-f9df62bf5bb5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.045609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 92/182] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:43.045609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283a524-687a-4435-a70e-f9df62bf5bb5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:43.050718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:43.055331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f7872-d887-47f1-b703-d2a926437b50 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:43.093299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab3ca9-93ec-4f81-b65c-186f00b8e80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:43.094100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab3ca9-93ec-4f81-b65c-186f00b8e80a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.094475 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab3ca9-93ec-4f81-b65c-186f00b8e80a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.094891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab3ca9-93ec-4f81-b65c-186f00b8e80a None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.096197 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab3ca9-93ec-4f81-b65c-186f00b8e80a None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:43.097855 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab3ca9-93ec-4f81-b65c-186f00b8e80a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:43.099854 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 91/183] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:42 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:43.105815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab3ca9-93ec-4f81-b65c-186f00b8e80a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:43.150275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:43.169653 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283a524-687a-4435-a70e-f9df62bf5bb5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:43.205655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cca1c2-35fe-44c2-af0b-0b6abac82d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:43.206361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cca1c2-35fe-44c2-af0b-0b6abac82d1c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.206644 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cca1c2-35fe-44c2-af0b-0b6abac82d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.206925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cca1c2-35fe-44c2-af0b-0b6abac82d1c None admin] PATH_INFO: `/v3/projects/576e6cec81664d5b903fbd4439ab041b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.208069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cca1c2-35fe-44c2-af0b-0b6abac82d1c None admin] RBAC: Authorizing `identity:get_project(project_id=576e6cec81664d5b903fbd4439ab041b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:43.209566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cca1c2-35fe-44c2-af0b-0b6abac82d1c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:43.209790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab3ca9-93ec-4f81-b65c-186f00b8e80a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:43.211257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 93/184] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:43 2022] GET /identity/v3/projects/576e6cec81664d5b903fbd4439ab041b => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:43.212368 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f991b8e-345b-40f0-ad8c-4b22bf6dbed1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.212647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f991b8e-345b-40f0-ad8c-4b22bf6dbed1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.212937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f991b8e-345b-40f0-ad8c-4b22bf6dbed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.231006 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8f991b8e-345b-40f0-ad8c-4b22bf6dbed1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:43.232204 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8f991b8e-345b-40f0-ad8c-4b22bf6dbed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:43.245948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3fcf2-3be2-4895-aed5-f5e90f481c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:43.246573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3fcf2-3be2-4895-aed5-f5e90f481c7d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.246857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3fcf2-3be2-4895-aed5-f5e90f481c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.247124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3fcf2-3be2-4895-aed5-f5e90f481c7d None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.247948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca3fcf2-3be2-4895-aed5-f5e90f481c7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:43.248259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca3fcf2-3be2-4895-aed5-f5e90f481c7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:43.249708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca3fcf2-3be2-4895-aed5-f5e90f481c7d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:43.263293 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 92/185] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:43.263293 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3fcf2-3be2-4895-aed5-f5e90f481c7d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:43.290801 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:43.300648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 94/186] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:43.301548 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f991b8e-345b-40f0-ad8c-4b22bf6dbed1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:43.376179 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3fcf2-3be2-4895-aed5-f5e90f481c7d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:43.394180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:43.413961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a351cc-5b9b-4531-bbba-7a6bead194f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:43.414604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a351cc-5b9b-4531-bbba-7a6bead194f2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.414882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a351cc-5b9b-4531-bbba-7a6bead194f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.415151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a351cc-5b9b-4531-bbba-7a6bead194f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.415884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a351cc-5b9b-4531-bbba-7a6bead194f2 None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e, user.id=42a19c5d3ad549648071d18e7300d6c1) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:43.416199 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a351cc-5b9b-4531-bbba-7a6bead194f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:43.417687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a351cc-5b9b-4531-bbba-7a6bead194f2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:43.418070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f991b8e-345b-40f0-ad8c-4b22bf6dbed1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:43.432934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 93/187] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:43 2022] GET /identity/v3/role_assignments?user.id=42a19c5d3ad549648071d18e7300d6c1&scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e&role.id=3b30504991194002b41f8587d0851c10 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:43.432934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46217903-04a4-472b-a72c-39fd3c8bcc0c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.432934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46217903-04a4-472b-a72c-39fd3c8bcc0c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.432934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46217903-04a4-472b-a72c-39fd3c8bcc0c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.432934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 94/188] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:43.432934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46217903-04a4-472b-a72c-39fd3c8bcc0c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:43.472739 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36211c19-3c88-4ef6-9665-405bdf497e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:43.475030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36211c19-3c88-4ef6-9665-405bdf497e56 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.475484 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36211c19-3c88-4ef6-9665-405bdf497e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.475908 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36211c19-3c88-4ef6-9665-405bdf497e56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.477404 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36211c19-3c88-4ef6-9665-405bdf497e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:43.479175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36211c19-3c88-4ef6-9665-405bdf497e56 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:43.487580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 95/189] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:43.487580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36211c19-3c88-4ef6-9665-405bdf497e56 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:43.554104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:43.567535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46217903-04a4-472b-a72c-39fd3c8bcc0c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:43.613344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fd407-3d95-4c25-85f3-d0b7d3d560a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:43.614083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fd407-3d95-4c25-85f3-d0b7d3d560a8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.614362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fd407-3d95-4c25-85f3-d0b7d3d560a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.614873 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fd407-3d95-4c25-85f3-d0b7d3d560a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.615483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33fd407-3d95-4c25-85f3-d0b7d3d560a8 None admin] RBAC: Adding query filter params (role.id=180edbb8a9e74555af89d6a7639f819a, scope.project.id=576e6cec81664d5b903fbd4439ab041b, user.id=40fbba8ecae846bca4e3b0fd6eaf8c85) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:43.615798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33fd407-3d95-4c25-85f3-d0b7d3d560a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:43.617526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33fd407-3d95-4c25-85f3-d0b7d3d560a8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:43.625932 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 95/190] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:43 2022] GET /identity/v3/role_assignments?user.id=40fbba8ecae846bca4e3b0fd6eaf8c85&scope.project.id=576e6cec81664d5b903fbd4439ab041b&role.id=180edbb8a9e74555af89d6a7639f819a => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:43.627184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e809d9-6e3c-4ddf-bce0-b2f5602ed52b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.627597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e809d9-6e3c-4ddf-bce0-b2f5602ed52b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.627734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e809d9-6e3c-4ddf-bce0-b2f5602ed52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.635285 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36211c19-3c88-4ef6-9665-405bdf497e56 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:43.656569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b5e809d9-6e3c-4ddf-bce0-b2f5602ed52b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:43.657927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b5e809d9-6e3c-4ddf-bce0-b2f5602ed52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:43.695890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcc4e1-7357-4390-84cc-0be034915a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:43.696555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcc4e1-7357-4390-84cc-0be034915a91 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.696863 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcc4e1-7357-4390-84cc-0be034915a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.697131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcc4e1-7357-4390-84cc-0be034915a91 None admin] PATH_INFO: `/v3/users/3917da18a49f40c2be38ac6025b032b6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.722860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcc4e1-7357-4390-84cc-0be034915a91 None admin] RBAC: Authorizing `identity:get_user(user_id=3917da18a49f40c2be38ac6025b032b6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:43.724834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcc4e1-7357-4390-84cc-0be034915a91 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:43.726824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 96/191] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:43 2022] GET /identity/v3/users/3917da18a49f40c2be38ac6025b032b6 => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:43.727798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcc4e1-7357-4390-84cc-0be034915a91 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:43.743145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:43.755323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 96/192] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:43.755323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e809d9-6e3c-4ddf-bce0-b2f5602ed52b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:43.860508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:43.882488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcc4e1-7357-4390-84cc-0be034915a91 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:43.883573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e809d9-6e3c-4ddf-bce0-b2f5602ed52b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:43.925457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77dde2c-b06c-45c6-b7f9-c3c824950d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:43.926115 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dde2c-b06c-45c6-b7f9-c3c824950d1d None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.926396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dde2c-b06c-45c6-b7f9-c3c824950d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.926669 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dde2c-b06c-45c6-b7f9-c3c824950d1d None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.927339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77dde2c-b06c-45c6-b7f9-c3c824950d1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:43.928837 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77dde2c-b06c-45c6-b7f9-c3c824950d1d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:43.930078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ddb7a-839e-460c-83f4-73e9e532ceda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:43.931008 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ddb7a-839e-460c-83f4-73e9e532ceda None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:43.931282 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ddb7a-839e-460c-83f4-73e9e532ceda None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:43.931552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ddb7a-839e-460c-83f4-73e9e532ceda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:43.943738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ddb7a-839e-460c-83f4-73e9e532ceda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-066ddb7a-839e-460c-83f4-73e9e532ceda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:43.949545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:43.950477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:43.951341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:43.951341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:43.951341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:43.951341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:43.951341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:43.951341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:43.951341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:43.951341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:43.951341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:43.951341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:43.951341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 20:59:43.951341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:43.951341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 97/193] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 20:59:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:43.951987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ddb7a-839e-460c-83f4-73e9e532ceda None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:43.962797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 97/194] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 20:59:43 2022] POST /identity/v3/users => generated 297 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:59:43.963487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77dde2c-b06c-45c6-b7f9-c3c824950d1d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:44.054775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:44.101121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ddb7a-839e-460c-83f4-73e9e532ceda None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:44.107568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77dde2c-b06c-45c6-b7f9-c3c824950d1d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:44.143183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158160e-3794-467d-a092-530c1be505c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:44.143809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158160e-3794-467d-a092-530c1be505c9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:44.144195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158160e-3794-467d-a092-530c1be505c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:44.144470 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158160e-3794-467d-a092-530c1be505c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:44.154036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddb0c3-5360-4380-ab6b-289c4a762a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:44.155502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddb0c3-5360-4380-ab6b-289c4a762a87 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:44.155502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddb0c3-5360-4380-ab6b-289c4a762a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:44.155502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddb0c3-5360-4380-ab6b-289c4a762a87 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:44.156000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddb0c3-5360-4380-ab6b-289c4a762a87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:44.156422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddb0c3-5360-4380-ab6b-289c4a762a87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:44.157851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddb0c3-5360-4380-ab6b-289c4a762a87 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:44.165949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:44.167143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 98/195] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 20:59:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:44.167667 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddb0c3-5360-4380-ab6b-289c4a762a87 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:44.233272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158160e-3794-467d-a092-530c1be505c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:44.234814 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158160e-3794-467d-a092-530c1be505c9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:44.289215 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddb0c3-5360-4380-ab6b-289c4a762a87 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:44.321200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:44.337041 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae4bb7-e01e-4c21-9c7e-f07f1fe4c79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:44.338004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae4bb7-e01e-4c21-9c7e-f07f1fe4c79d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:44.338286 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae4bb7-e01e-4c21-9c7e-f07f1fe4c79d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:44.338562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae4bb7-e01e-4c21-9c7e-f07f1fe4c79d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:44.350681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae4bb7-e01e-4c21-9c7e-f07f1fe4c79d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9cae4bb7-e01e-4c21-9c7e-f07f1fe4c79d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:44.356189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:44.357158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:44.357952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:44.357952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:44.357952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:44.357952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:44.357952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:44.357952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:44.357952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:44.357952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:44.357952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:44.357952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:44.357952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 20:59:44.357952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:44.357952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 99/196] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:44.358588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae4bb7-e01e-4c21-9c7e-f07f1fe4c79d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:44.371412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 98/197] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:44.372574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158160e-3794-467d-a092-530c1be505c9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:44.429032 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:44.482857 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae4bb7-e01e-4c21-9c7e-f07f1fe4c79d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:44.489162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158160e-3794-467d-a092-530c1be505c9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:44.519722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f354ff-177e-4d31-931a-1e03a51c2706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:44.520376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f354ff-177e-4d31-931a-1e03a51c2706 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:44.520671 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f354ff-177e-4d31-931a-1e03a51c2706 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:44.520967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f354ff-177e-4d31-931a-1e03a51c2706 None admin] PATH_INFO: `/v3/users/3917da18a49f40c2be38ac6025b032b6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:44.522245 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f354ff-177e-4d31-931a-1e03a51c2706 None admin] RBAC: Authorizing `identity:get_user(user_id=3917da18a49f40c2be38ac6025b032b6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:44.523711 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f354ff-177e-4d31-931a-1e03a51c2706 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:44.525343 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 100/198] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:44 2022] GET /identity/v3/users/3917da18a49f40c2be38ac6025b032b6 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:44.544204 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f354ff-177e-4d31-931a-1e03a51c2706 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:44.547426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a654e-57e5-4335-b84a-3b7870e06c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:44.548072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a654e-57e5-4335-b84a-3b7870e06c8e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:44.548349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a654e-57e5-4335-b84a-3b7870e06c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:44.548619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a654e-57e5-4335-b84a-3b7870e06c8e None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:44.549338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a654e-57e5-4335-b84a-3b7870e06c8e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:44.549640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a654e-57e5-4335-b84a-3b7870e06c8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:44.551083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a654e-57e5-4335-b84a-3b7870e06c8e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:44.563310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 99/199] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:44.568769 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a654e-57e5-4335-b84a-3b7870e06c8e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:44.617620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:44.691634 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f354ff-177e-4d31-931a-1e03a51c2706 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:44.696423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a654e-57e5-4335-b84a-3b7870e06c8e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:44.706912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:44.735713 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85bf005-e6c8-40c9-8e6d-5bc02cd7bb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:44.736354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bf005-e6c8-40c9-8e6d-5bc02cd7bb0c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:44.736632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bf005-e6c8-40c9-8e6d-5bc02cd7bb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:44.737007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bf005-e6c8-40c9-8e6d-5bc02cd7bb0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:44.739550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652126d-dbb6-4464-be07-3dd562791ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:44.740597 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652126d-dbb6-4464-be07-3dd562791ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:44.741170 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652126d-dbb6-4464-be07-3dd562791ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:44.741602 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652126d-dbb6-4464-be07-3dd562791ef6 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:44.742946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652126d-dbb6-4464-be07-3dd562791ef6 None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:44.744911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652126d-dbb6-4464-be07-3dd562791ef6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:44.746871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 101/200] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:44 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:44.747836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652126d-dbb6-4464-be07-3dd562791ef6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:44.820130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85bf005-e6c8-40c9-8e6d-5bc02cd7bb0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:44.821521 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85bf005-e6c8-40c9-8e6d-5bc02cd7bb0c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:44.869465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652126d-dbb6-4464-be07-3dd562791ef6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:44.891911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:44.909591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61513a-eb6c-467e-afdd-1d29e5c8edea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:44.910614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61513a-eb6c-467e-afdd-1d29e5c8edea None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:44.911054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61513a-eb6c-467e-afdd-1d29e5c8edea None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:44.911482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61513a-eb6c-467e-afdd-1d29e5c8edea None admin] PATH_INFO: `/v3/users/72a9afd5d53e4c1a97376bda65fbdd64` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:44.912853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61513a-eb6c-467e-afdd-1d29e5c8edea None admin] RBAC: Authorizing `identity:get_user(user_id=72a9afd5d53e4c1a97376bda65fbdd64)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:44.914579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61513a-eb6c-467e-afdd-1d29e5c8edea None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:44.916526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 102/201] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:44 2022] GET /identity/v3/users/72a9afd5d53e4c1a97376bda65fbdd64 => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:44.918126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61513a-eb6c-467e-afdd-1d29e5c8edea None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:44.927028 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 100/202] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:44.927889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85bf005-e6c8-40c9-8e6d-5bc02cd7bb0c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:44.939096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:45.041451 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61513a-eb6c-467e-afdd-1d29e5c8edea None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:45.046821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85bf005-e6c8-40c9-8e6d-5bc02cd7bb0c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:45.080827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb357fdc-99ed-442c-928e-b633ca7c0e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:45.081824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb357fdc-99ed-442c-928e-b633ca7c0e35 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:45.082299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb357fdc-99ed-442c-928e-b633ca7c0e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:45.082767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb357fdc-99ed-442c-928e-b633ca7c0e35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:45.083680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb357fdc-99ed-442c-928e-b633ca7c0e35 None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=402258177cb849268ec3155d32bedd84, user.id=3917da18a49f40c2be38ac6025b032b6) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:45.083920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e838a8-90f6-42ea-9877-69b5e098a9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:45.084393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb357fdc-99ed-442c-928e-b633ca7c0e35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:45.084560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e838a8-90f6-42ea-9877-69b5e098a9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:45.084846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e838a8-90f6-42ea-9877-69b5e098a9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:45.085179 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e838a8-90f6-42ea-9877-69b5e098a9c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:45.086647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb357fdc-99ed-442c-928e-b633ca7c0e35 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:45.093397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 103/203] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:44 2022] GET /identity/v3/role_assignments?user.id=3917da18a49f40c2be38ac6025b032b6&scope.project.id=402258177cb849268ec3155d32bedd84&role.id=3b30504991194002b41f8587d0851c10 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:45.094577 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb357fdc-99ed-442c-928e-b633ca7c0e35 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:45.187503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:45.224389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e838a8-90f6-42ea-9877-69b5e098a9c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:45.225935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e838a8-90f6-42ea-9877-69b5e098a9c0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:45.236894 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:45.250492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb357fdc-99ed-442c-928e-b633ca7c0e35 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:45.298917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547f33b-1e6d-41fd-bfea-84fcd4904659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:45.299927 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547f33b-1e6d-41fd-bfea-84fcd4904659 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:45.300461 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547f33b-1e6d-41fd-bfea-84fcd4904659 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:45.301026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547f33b-1e6d-41fd-bfea-84fcd4904659 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:45.305445 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547f33b-1e6d-41fd-bfea-84fcd4904659 None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:45.307548 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547f33b-1e6d-41fd-bfea-84fcd4904659 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:45.310026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 104/204] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:45 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:45.315918 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547f33b-1e6d-41fd-bfea-84fcd4904659 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:45.338405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 101/205] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:45.341020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e838a8-90f6-42ea-9877-69b5e098a9c0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:45.449665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547f33b-1e6d-41fd-bfea-84fcd4904659 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:45.455844 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e838a8-90f6-42ea-9877-69b5e098a9c0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:45.471752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:45.474934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:45.500690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f719a-2f60-40dd-9242-e33358c72539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:45.502780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f719a-2f60-40dd-9242-e33358c72539 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:45.503404 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f719a-2f60-40dd-9242-e33358c72539 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:45.504844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f719a-2f60-40dd-9242-e33358c72539 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:45.505942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f719a-2f60-40dd-9242-e33358c72539 None admin] RBAC: Adding query filter params (role.id=e945f087a5bc4c38bd8848782f76c214, scope.project.id=402258177cb849268ec3155d32bedd84, user.id=3917da18a49f40c2be38ac6025b032b6) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:45.507162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f719a-2f60-40dd-9242-e33358c72539 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:45.509461 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f719a-2f60-40dd-9242-e33358c72539 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:45.518720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 105/206] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:45 2022] GET /identity/v3/role_assignments?user.id=3917da18a49f40c2be38ac6025b032b6&scope.project.id=402258177cb849268ec3155d32bedd84&role.id=e945f087a5bc4c38bd8848782f76c214 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:45.528442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e7793-d21d-4294-921a-a1b2a3ac539f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:45.529748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e7793-d21d-4294-921a-a1b2a3ac539f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:45.530230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e7793-d21d-4294-921a-a1b2a3ac539f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:45.530723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e7793-d21d-4294-921a-a1b2a3ac539f None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:45.537700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e7793-d21d-4294-921a-a1b2a3ac539f None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:45.539618 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e7793-d21d-4294-921a-a1b2a3ac539f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:45.542507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 102/207] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:45 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:45.548555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f719a-2f60-40dd-9242-e33358c72539 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:45.704915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f719a-2f60-40dd-9242-e33358c72539 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:45.748400 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53aa7c-975b-488f-a321-4ac5d21ffd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:45.749057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53aa7c-975b-488f-a321-4ac5d21ffd98 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:45.749329 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53aa7c-975b-488f-a321-4ac5d21ffd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:45.749594 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53aa7c-975b-488f-a321-4ac5d21ffd98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:45.750333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53aa7c-975b-488f-a321-4ac5d21ffd98 None admin] RBAC: Adding query filter params (role.id=6abadafbec7f43d2b2cc2df439c44eff, scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e, user.id=72a9afd5d53e4c1a97376bda65fbdd64) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:45.750632 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53aa7c-975b-488f-a321-4ac5d21ffd98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:45.752740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53aa7c-975b-488f-a321-4ac5d21ffd98 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:45.758746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 106/208] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:45 2022] GET /identity/v3/role_assignments?user.id=72a9afd5d53e4c1a97376bda65fbdd64&scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e&role.id=6abadafbec7f43d2b2cc2df439c44eff => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:46.454001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3739d7d-39a4-4154-8418-8df5dc91ab1e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:46.454391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3739d7d-39a4-4154-8418-8df5dc91ab1e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:46.454560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3739d7d-39a4-4154-8418-8df5dc91ab1e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:46.456176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 103/209] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:46.462107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49f281-7c9a-44b0-a80d-fda3ad22a61e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:46.462395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49f281-7c9a-44b0-a80d-fda3ad22a61e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:46.462652 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49f281-7c9a-44b0-a80d-fda3ad22a61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:46.485653 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0b49f281-7c9a-44b0-a80d-fda3ad22a61e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:46.486884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0b49f281-7c9a-44b0-a80d-fda3ad22a61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:46.539098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:46.567400 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 107/210] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:46.681766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55211f-f780-4fa9-834f-c50bd96952d0 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:46.682054 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55211f-f780-4fa9-834f-c50bd96952d0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:46.682360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55211f-f780-4fa9-834f-c50bd96952d0 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:46.683834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 104/211] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:46.689609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160e00b-d764-43b0-9109-6a251f31220e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:46.689897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160e00b-d764-43b0-9109-6a251f31220e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:46.690172 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160e00b-d764-43b0-9109-6a251f31220e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:46.709079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d160e00b-d764-43b0-9109-6a251f31220e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:46.710195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d160e00b-d764-43b0-9109-6a251f31220e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:46.781283 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 108/212] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:46.853969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55211f-f780-4fa9-834f-c50bd96952d0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:46.964970 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55211f-f780-4fa9-834f-c50bd96952d0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:47.008000 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc3cce-b9ae-4e30-83cd-85236c289373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:47.009110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc3cce-b9ae-4e30-83cd-85236c289373 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:47.009608 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc3cce-b9ae-4e30-83cd-85236c289373 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:47.010087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc3cce-b9ae-4e30-83cd-85236c289373 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:47.024197 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc3cce-b9ae-4e30-83cd-85236c289373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-75fc3cce-b9ae-4e30-83cd-85236c289373 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:47.030516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:47.031641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:47.032432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:47.032432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:47.032432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:47.032432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:47.032432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:47.032432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:47.032432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:47.032432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:47.032432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:47.032432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:47.032432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:47.032432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:47.035326 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 105/213] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:47.042405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160e00b-d764-43b0-9109-6a251f31220e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:47.061443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc3cce-b9ae-4e30-83cd-85236c289373 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:47.103923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:47.163197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160e00b-d764-43b0-9109-6a251f31220e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:47.177290 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc3cce-b9ae-4e30-83cd-85236c289373 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:47.197964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:47.201617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b2d5a-905f-427b-a3ec-8c197e9f436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:47.202416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b2d5a-905f-427b-a3ec-8c197e9f436d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:47.202890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b2d5a-905f-427b-a3ec-8c197e9f436d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:47.203168 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b2d5a-905f-427b-a3ec-8c197e9f436d None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:47.203947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b2d5a-905f-427b-a3ec-8c197e9f436d None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:47.204346 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b2d5a-905f-427b-a3ec-8c197e9f436d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:47.205815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b2d5a-905f-427b-a3ec-8c197e9f436d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:47.215523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574b34d-dd74-4efe-b6e1-80b2c1fd6c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:47.215693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 109/214] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:47.215922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574b34d-dd74-4efe-b6e1-80b2c1fd6c2e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:47.216096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574b34d-dd74-4efe-b6e1-80b2c1fd6c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:47.216432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574b34d-dd74-4efe-b6e1-80b2c1fd6c2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:47.217593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574b34d-dd74-4efe-b6e1-80b2c1fd6c2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:47.219059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574b34d-dd74-4efe-b6e1-80b2c1fd6c2e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:47.220990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b2d5a-905f-427b-a3ec-8c197e9f436d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:47.221119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 106/215] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:47.226296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574b34d-dd74-4efe-b6e1-80b2c1fd6c2e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:47.333433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574b34d-dd74-4efe-b6e1-80b2c1fd6c2e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:47.336424 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:47.341798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b2d5a-905f-427b-a3ec-8c197e9f436d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:47.373923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8ab5c-7486-4757-834c-27a34335c805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:47.374581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8ab5c-7486-4757-834c-27a34335c805 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:47.374972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8ab5c-7486-4757-834c-27a34335c805 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:47.375531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8ab5c-7486-4757-834c-27a34335c805 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:47.376088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8ab5c-7486-4757-834c-27a34335c805 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:47.377887 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8ab5c-7486-4757-834c-27a34335c805 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:47.380753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6cd12-0636-4c83-a180-8f392dc8b65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:47.381593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6cd12-0636-4c83-a180-8f392dc8b65d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:47.382040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6cd12-0636-4c83-a180-8f392dc8b65d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:47.382502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6cd12-0636-4c83-a180-8f392dc8b65d None admin] PATH_INFO: `/v3/users/252526882e334874a5173783d27d41c6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:47.400016 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6cd12-0636-4c83-a180-8f392dc8b65d None admin] RBAC: Authorizing `identity:get_user(user_id=252526882e334874a5173783d27d41c6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:47.402675 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6cd12-0636-4c83-a180-8f392dc8b65d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:47.403226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 107/216] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 20:59:47 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:59:47.409052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 110/217] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:47 2022] GET /identity/v3/users/252526882e334874a5173783d27d41c6 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:47.414546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8ab5c-7486-4757-834c-27a34335c805 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:47.547948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8ab5c-7486-4757-834c-27a34335c805 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:47.592421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ba1b4-d204-4aba-82d3-99c9f8a0807e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:47.593489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ba1b4-d204-4aba-82d3-99c9f8a0807e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:47.594036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ba1b4-d204-4aba-82d3-99c9f8a0807e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:47.594529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ba1b4-d204-4aba-82d3-99c9f8a0807e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:47.629501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:47.644201 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcd4b2-b9cf-4074-b622-92ee6b4cb3b7 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:47.645184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcd4b2-b9cf-4074-b622-92ee6b4cb3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:47.645647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcd4b2-b9cf-4074-b622-92ee6b4cb3b7 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:47.647498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 111/218] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:59:47.654211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f3c07-c75d-4471-b17a-72f90be3e9d6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:47.654799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f3c07-c75d-4471-b17a-72f90be3e9d6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:47.655824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f3c07-c75d-4471-b17a-72f90be3e9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:47.672702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ba1b4-d204-4aba-82d3-99c9f8a0807e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:47.674222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ba1b4-d204-4aba-82d3-99c9f8a0807e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:47.677106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4b1f3c07-c75d-4471-b17a-72f90be3e9d6 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:47.678570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4b1f3c07-c75d-4471-b17a-72f90be3e9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:47.715165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:47.787255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 112/219] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:47.790986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 108/220] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:47.796581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f3c07-c75d-4471-b17a-72f90be3e9d6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:47.914047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f3c07-c75d-4471-b17a-72f90be3e9d6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:47.952268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120228f4-5a81-4b43-9457-dcb8ad499c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:47.952949 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120228f4-5a81-4b43-9457-dcb8ad499c55 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:47.953223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120228f4-5a81-4b43-9457-dcb8ad499c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:47.953498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120228f4-5a81-4b43-9457-dcb8ad499c55 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:47.954640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120228f4-5a81-4b43-9457-dcb8ad499c55 None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:47.956106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120228f4-5a81-4b43-9457-dcb8ad499c55 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:47.957972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 113/221] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:47 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:47.963813 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ba1b4-d204-4aba-82d3-99c9f8a0807e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:48.031809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:48.041300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98502ed2-8d49-4dc6-9aab-a52c29d77b41 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:48.041590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98502ed2-8d49-4dc6-9aab-a52c29d77b41 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:48.041897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98502ed2-8d49-4dc6-9aab-a52c29d77b41 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:48.043494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 114/222] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:59:48.058394 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b8de5-836c-4c9e-8541-a7fd108d2ee2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:48.058688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b8de5-836c-4c9e-8541-a7fd108d2ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:48.058948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b8de5-836c-4c9e-8541-a7fd108d2ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:48.085566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e22b8de5-836c-4c9e-8541-a7fd108d2ee2 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:48.086781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e22b8de5-836c-4c9e-8541-a7fd108d2ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:48.101237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ba1b4-d204-4aba-82d3-99c9f8a0807e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:48.109021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:48.146084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661abe9b-a989-46ab-a342-a4a1322940cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:48.146773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661abe9b-a989-46ab-a342-a4a1322940cb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:48.147375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661abe9b-a989-46ab-a342-a4a1322940cb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:48.147656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661abe9b-a989-46ab-a342-a4a1322940cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:48.148914 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661abe9b-a989-46ab-a342-a4a1322940cb None admin] RBAC: Adding query filter params (role.id=180edbb8a9e74555af89d6a7639f819a, scope.project.id=402258177cb849268ec3155d32bedd84, user.id=252526882e334874a5173783d27d41c6) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:48.149196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661abe9b-a989-46ab-a342-a4a1322940cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:48.150699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661abe9b-a989-46ab-a342-a4a1322940cb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:48.164784 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 109/223] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:47 2022] GET /identity/v3/role_assignments?user.id=252526882e334874a5173783d27d41c6&scope.project.id=402258177cb849268ec3155d32bedd84&role.id=180edbb8a9e74555af89d6a7639f819a => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:48.164784 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661abe9b-a989-46ab-a342-a4a1322940cb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:48.181262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 115/224] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 06 20:59:48.277950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661abe9b-a989-46ab-a342-a4a1322940cb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:48.304936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:48.323630 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d034e69-3f68-4303-81f2-7e76e4043ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:48.324299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d034e69-3f68-4303-81f2-7e76e4043ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:48.324579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d034e69-3f68-4303-81f2-7e76e4043ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:48.324878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d034e69-3f68-4303-81f2-7e76e4043ff4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:48.342171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d034e69-3f68-4303-81f2-7e76e4043ff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6d034e69-3f68-4303-81f2-7e76e4043ff4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:48.347561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:48.348449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:48.349291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:48.349291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:48.349291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:48.349291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:48.349291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:48.349291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:48.349291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:48.349291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:48.349291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:48.349291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:48.349291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:48.349291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:48.349291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 110/225] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 20:59:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:48.355008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b8de5-836c-4c9e-8541-a7fd108d2ee2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:48.511489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b8de5-836c-4c9e-8541-a7fd108d2ee2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:48.561808 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732bcbc-75d8-43ec-a0d4-229ce75218cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:48.562159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d034e69-3f68-4303-81f2-7e76e4043ff4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:48.562614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732bcbc-75d8-43ec-a0d4-229ce75218cf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:48.562691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732bcbc-75d8-43ec-a0d4-229ce75218cf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:48.562966 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732bcbc-75d8-43ec-a0d4-229ce75218cf None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:48.563650 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732bcbc-75d8-43ec-a0d4-229ce75218cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:48.563953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732bcbc-75d8-43ec-a0d4-229ce75218cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:48.565676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732bcbc-75d8-43ec-a0d4-229ce75218cf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:48.578183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 116/226] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:48.594232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732bcbc-75d8-43ec-a0d4-229ce75218cf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:48.632265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:48.763070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d034e69-3f68-4303-81f2-7e76e4043ff4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:48.772936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732bcbc-75d8-43ec-a0d4-229ce75218cf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:48.822575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158c3a5-e2c3-43b9-b9b8-c4b6eaf6774a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:48.823216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158c3a5-e2c3-43b9-b9b8-c4b6eaf6774a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:48.823491 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158c3a5-e2c3-43b9-b9b8-c4b6eaf6774a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:48.823762 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158c3a5-e2c3-43b9-b9b8-c4b6eaf6774a None admin] PATH_INFO: `/v3/users/3917da18a49f40c2be38ac6025b032b6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:48.825647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158c3a5-e2c3-43b9-b9b8-c4b6eaf6774a None admin] RBAC: Authorizing `identity:get_user(user_id=3917da18a49f40c2be38ac6025b032b6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:48.827103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158c3a5-e2c3-43b9-b9b8-c4b6eaf6774a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:48.829887 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 117/227] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:48 2022] GET /identity/v3/users/3917da18a49f40c2be38ac6025b032b6 => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:48.833819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e930a58-5550-4d59-851b-34e0eaf6e6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:48.834468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e930a58-5550-4d59-851b-34e0eaf6e6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:48.834744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e930a58-5550-4d59-851b-34e0eaf6e6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:48.835009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e930a58-5550-4d59-851b-34e0eaf6e6d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:48.836281 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158c3a5-e2c3-43b9-b9b8-c4b6eaf6774a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:48.857850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e930a58-5550-4d59-851b-34e0eaf6e6d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8e930a58-5550-4d59-851b-34e0eaf6e6d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:48.864962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:48.865820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:48.866611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:48.866611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:48.866611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:48.866611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:48.866611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:48.866611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:48.866611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:48.866611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:48.866611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:48.866611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:48.866611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 20:59:48.866611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:48.872980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 111/228] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 20:59:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:59:48.878852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e930a58-5550-4d59-851b-34e0eaf6e6d4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:48.944888 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:48.962383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158c3a5-e2c3-43b9-b9b8-c4b6eaf6774a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:48.984994 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:48.999069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be030d22-d673-47f5-a6c9-bd4ee32f87a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:48.999710 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be030d22-d673-47f5-a6c9-bd4ee32f87a5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:48.999987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be030d22-d673-47f5-a6c9-bd4ee32f87a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:49.000257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be030d22-d673-47f5-a6c9-bd4ee32f87a5 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:49.001506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be030d22-d673-47f5-a6c9-bd4ee32f87a5 None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:49.002984 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be030d22-d673-47f5-a6c9-bd4ee32f87a5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:49.004687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 118/229] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:48 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:49.010592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be030d22-d673-47f5-a6c9-bd4ee32f87a5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:49.117561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be030d22-d673-47f5-a6c9-bd4ee32f87a5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:49.117918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e930a58-5550-4d59-851b-34e0eaf6e6d4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:49.156472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eeead5-3184-4163-b338-31434dbc89c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:49.157184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eeead5-3184-4163-b338-31434dbc89c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:49.157462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eeead5-3184-4163-b338-31434dbc89c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:49.157737 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eeead5-3184-4163-b338-31434dbc89c8 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84/users/3917da18a49f40c2be38ac6025b032b6/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:49.163522 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eeead5-3184-4163-b338-31434dbc89c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=402258177cb849268ec3155d32bedd84, user_id=3917da18a49f40c2be38ac6025b032b6, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:49.165024 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eeead5-3184-4163-b338-31434dbc89c8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:49.187059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 119/230] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 20:59:49 2022] PUT /identity/v3/projects/402258177cb849268ec3155d32bedd84/users/3917da18a49f40c2be38ac6025b032b6/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:59:49.190426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416805d-fa95-4b38-9b8f-f208233e00e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:49.191392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416805d-fa95-4b38-9b8f-f208233e00e7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:49.191926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416805d-fa95-4b38-9b8f-f208233e00e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:49.192398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416805d-fa95-4b38-9b8f-f208233e00e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:49.193531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416805d-fa95-4b38-9b8f-f208233e00e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:49.194053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416805d-fa95-4b38-9b8f-f208233e00e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:49.195743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416805d-fa95-4b38-9b8f-f208233e00e7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:49.213453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 112/231] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 20:59:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:49.220175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eeead5-3184-4163-b338-31434dbc89c8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:49.245312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:49.362360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eeead5-3184-4163-b338-31434dbc89c8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:49.412874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525fe68d-a92a-4010-8d09-43bf9a4237da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:49.413443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fe68d-a92a-4010-8d09-43bf9a4237da None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:49.413717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fe68d-a92a-4010-8d09-43bf9a4237da None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:49.413982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fe68d-a92a-4010-8d09-43bf9a4237da None admin] PATH_INFO: `/v3/users/3917da18a49f40c2be38ac6025b032b6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:49.415104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525fe68d-a92a-4010-8d09-43bf9a4237da None admin] RBAC: Authorizing `identity:get_user(user_id=3917da18a49f40c2be38ac6025b032b6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:49.416823 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525fe68d-a92a-4010-8d09-43bf9a4237da None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:49.418650 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 120/232] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:49 2022] GET /identity/v3/users/3917da18a49f40c2be38ac6025b032b6 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:49.424389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416805d-fa95-4b38-9b8f-f208233e00e7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:49.532344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416805d-fa95-4b38-9b8f-f208233e00e7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:49.539957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:49.569272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55774cdc-3437-4454-8751-5773f4aa28ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:49.569943 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55774cdc-3437-4454-8751-5773f4aa28ad None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:49.570177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55774cdc-3437-4454-8751-5773f4aa28ad None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:49.570508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55774cdc-3437-4454-8751-5773f4aa28ad None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:49.571698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55774cdc-3437-4454-8751-5773f4aa28ad None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:49.573205 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55774cdc-3437-4454-8751-5773f4aa28ad None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:49.574860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 113/233] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:49 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:49.581055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525fe68d-a92a-4010-8d09-43bf9a4237da None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:49.676422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:49.696268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525fe68d-a92a-4010-8d09-43bf9a4237da None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:49.731746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5bb12-b495-4fe2-99c2-1511e948c0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:49.732449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5bb12-b495-4fe2-99c2-1511e948c0d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:49.732750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5bb12-b495-4fe2-99c2-1511e948c0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:49.733023 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5bb12-b495-4fe2-99c2-1511e948c0d0 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84/users/3917da18a49f40c2be38ac6025b032b6/roles/e945f087a5bc4c38bd8848782f76c214` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:49.735190 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5bb12-b495-4fe2-99c2-1511e948c0d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=402258177cb849268ec3155d32bedd84, user_id=3917da18a49f40c2be38ac6025b032b6, role_id=e945f087a5bc4c38bd8848782f76c214)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:49.736644 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5bb12-b495-4fe2-99c2-1511e948c0d0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:49.756446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 121/234] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 20:59:49 2022] PUT /identity/v3/projects/402258177cb849268ec3155d32bedd84/users/3917da18a49f40c2be38ac6025b032b6/roles/e945f087a5bc4c38bd8848782f76c214 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:59:50.316879 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1e91a-9a06-40ce-abd8-ba22221ee3eb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:50.317257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1e91a-9a06-40ce-abd8-ba22221ee3eb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:50.317448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1e91a-9a06-40ce-abd8-ba22221ee3eb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:50.319210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 114/235] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:50.332477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd3656-65c5-44b1-8e52-3654a7617bca None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:50.333437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd3656-65c5-44b1-8e52-3654a7617bca None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:50.333898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd3656-65c5-44b1-8e52-3654a7617bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:50.361753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dacd3656-65c5-44b1-8e52-3654a7617bca None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:50.363809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dacd3656-65c5-44b1-8e52-3654a7617bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:50.469761 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 122/236] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:50.481892 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271ed8c-d7fe-4d62-a145-f65e2cf8817d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:50.482249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271ed8c-d7fe-4d62-a145-f65e2cf8817d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:50.482510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271ed8c-d7fe-4d62-a145-f65e2cf8817d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:50.485998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 115/237] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:50.498606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bc21c-3e5a-4c03-8a0c-21cee429d8f2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:50.499177 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bc21c-3e5a-4c03-8a0c-21cee429d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:50.499640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bc21c-3e5a-4c03-8a0c-21cee429d8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:50.533998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e39bc21c-3e5a-4c03-8a0c-21cee429d8f2 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:50.535221 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e39bc21c-3e5a-4c03-8a0c-21cee429d8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:50.621852 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 123/238] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:50.746108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271ed8c-d7fe-4d62-a145-f65e2cf8817d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:50.885926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271ed8c-d7fe-4d62-a145-f65e2cf8817d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:50.896125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03261d7-1cca-4a74-8b28-03bc283e942f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:50.896407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03261d7-1cca-4a74-8b28-03bc283e942f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:50.896917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03261d7-1cca-4a74-8b28-03bc283e942f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:50.898803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 124/239] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:50.905031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f5fe-bd55-4edf-a60e-75efe1aff371 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:50.905382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f5fe-bd55-4edf-a60e-75efe1aff371 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:50.905727 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f5fe-bd55-4edf-a60e-75efe1aff371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:50.913079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:50.940593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1059f5fe-bd55-4edf-a60e-75efe1aff371 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:50.942940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1059f5fe-bd55-4edf-a60e-75efe1aff371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:50.957907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d7a02-779b-4f7a-8522-4300ed2c366a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:50.958884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d7a02-779b-4f7a-8522-4300ed2c366a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:50.959377 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d7a02-779b-4f7a-8522-4300ed2c366a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:50.959830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d7a02-779b-4f7a-8522-4300ed2c366a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:50.961157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d7a02-779b-4f7a-8522-4300ed2c366a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:50.962838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d7a02-779b-4f7a-8522-4300ed2c366a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:50.967233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 116/240] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:50.973215 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d7a02-779b-4f7a-8522-4300ed2c366a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:51.066302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:51.082080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 125/241] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:51.083576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059f5fe-bd55-4edf-a60e-75efe1aff371 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:51.187401 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d7a02-779b-4f7a-8522-4300ed2c366a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:51.239545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa29883-ee79-4e8b-a82f-7195edd60c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:51.240917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa29883-ee79-4e8b-a82f-7195edd60c6d None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:51.241526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa29883-ee79-4e8b-a82f-7195edd60c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:51.242673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa29883-ee79-4e8b-a82f-7195edd60c6d None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:51.243880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa29883-ee79-4e8b-a82f-7195edd60c6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:51.246933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa29883-ee79-4e8b-a82f-7195edd60c6d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:51.283301 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059f5fe-bd55-4edf-a60e-75efe1aff371 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:51.287706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 117/242] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 20:59:50 2022] POST /identity/v3/users => generated 293 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:59:51.322072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:51.329780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256b16a-89fd-48cb-84a5-87b8044a24b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:51.330793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256b16a-89fd-48cb-84a5-87b8044a24b2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:51.331334 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256b16a-89fd-48cb-84a5-87b8044a24b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:51.331789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256b16a-89fd-48cb-84a5-87b8044a24b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:51.345539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256b16a-89fd-48cb-84a5-87b8044a24b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-5256b16a-89fd-48cb-84a5-87b8044a24b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:51.351545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:51.352413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:51.353228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:51.353228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:51.353228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:51.353228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:51.353228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:51.353228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:51.353228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:51.353228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:51.353228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:51.353228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:51.353228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 20:59:51.353228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:51.355655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 126/243] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:51.362137 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa29883-ee79-4e8b-a82f-7195edd60c6d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:51.406239 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256b16a-89fd-48cb-84a5-87b8044a24b2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:51.560144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:51.564360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa29883-ee79-4e8b-a82f-7195edd60c6d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:51.598718 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256b16a-89fd-48cb-84a5-87b8044a24b2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:51.631387 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5a406-9391-4025-ac4a-b69c03d3ed4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:51.632406 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a406-9391-4025-ac4a-b69c03d3ed4d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:51.633012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a406-9391-4025-ac4a-b69c03d3ed4d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:51.633507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a406-9391-4025-ac4a-b69c03d3ed4d None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:51.634496 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5a406-9391-4025-ac4a-b69c03d3ed4d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:51.634988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5a406-9391-4025-ac4a-b69c03d3ed4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:51.636680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5a406-9391-4025-ac4a-b69c03d3ed4d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:51.643067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927252ad-8025-45dc-938e-77037f9572fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:51.643734 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927252ad-8025-45dc-938e-77037f9572fe None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:51.644525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927252ad-8025-45dc-938e-77037f9572fe None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:51.644525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927252ad-8025-45dc-938e-77037f9572fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:51.648634 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 118/244] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:51.657862 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5a406-9391-4025-ac4a-b69c03d3ed4d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:51.659612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927252ad-8025-45dc-938e-77037f9572fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-927252ad-8025-45dc-938e-77037f9572fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:51.667218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:51.668148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:51.669060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:51.669060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:51.669060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:51.669060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:51.669060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:51.669060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:51.669060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:51.669060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:51.669060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:51.669060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:51.669060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:51.669060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:51.669060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 127/245] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:59:51.676862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927252ad-8025-45dc-938e-77037f9572fe None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:51.749210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:51.781676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5a406-9391-4025-ac4a-b69c03d3ed4d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:51.813841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927252ad-8025-45dc-938e-77037f9572fe None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:51.818933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411aa4f-b46d-4ea0-a15c-59aeb46a1745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:51.819590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411aa4f-b46d-4ea0-a15c-59aeb46a1745 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:51.819867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411aa4f-b46d-4ea0-a15c-59aeb46a1745 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:51.820134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411aa4f-b46d-4ea0-a15c-59aeb46a1745 None admin] PATH_INFO: `/v3/users/c4a0d511d1bb4d66b636fe1651821f21` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:51.835507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411aa4f-b46d-4ea0-a15c-59aeb46a1745 None admin] RBAC: Authorizing `identity:get_user(user_id=c4a0d511d1bb4d66b636fe1651821f21)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:51.836554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411aa4f-b46d-4ea0-a15c-59aeb46a1745 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:51.838697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 119/246] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:51 2022] GET /identity/v3/users/c4a0d511d1bb4d66b636fe1651821f21 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:51.846826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411aa4f-b46d-4ea0-a15c-59aeb46a1745 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:51.871988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:51.874329 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2950013-fbd1-4492-9c0b-7a1791988530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:51.875268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2950013-fbd1-4492-9c0b-7a1791988530 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:51.876606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2950013-fbd1-4492-9c0b-7a1791988530 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:51.877254 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2950013-fbd1-4492-9c0b-7a1791988530 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:51.878232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2950013-fbd1-4492-9c0b-7a1791988530 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:51.878828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2950013-fbd1-4492-9c0b-7a1791988530 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:51.881511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2950013-fbd1-4492-9c0b-7a1791988530 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:51.894271 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 128/247] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:51.901878 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2950013-fbd1-4492-9c0b-7a1791988530 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:51.967463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411aa4f-b46d-4ea0-a15c-59aeb46a1745 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:52.005524 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118e6a0-eab4-4f66-89e3-d1334e8f5b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:52.006153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118e6a0-eab4-4f66-89e3-d1334e8f5b31 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:52.006427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118e6a0-eab4-4f66-89e3-d1334e8f5b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:52.006696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118e6a0-eab4-4f66-89e3-d1334e8f5b31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:52.013286 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2950013-fbd1-4492-9c0b-7a1791988530 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:52.028515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:52.058384 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9e19a-b85e-4dba-b89f-69e0913521e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:52.059047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9e19a-b85e-4dba-b89f-69e0913521e4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:52.059328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9e19a-b85e-4dba-b89f-69e0913521e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:52.059598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9e19a-b85e-4dba-b89f-69e0913521e4 None admin] PATH_INFO: `/v3/users/252526882e334874a5173783d27d41c6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:52.061847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9e19a-b85e-4dba-b89f-69e0913521e4 None admin] RBAC: Authorizing `identity:get_user(user_id=252526882e334874a5173783d27d41c6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:52.063929 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9e19a-b85e-4dba-b89f-69e0913521e4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:52.066411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 129/248] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:51 2022] GET /identity/v3/users/252526882e334874a5173783d27d41c6 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:52.068512 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229793c-778c-4fec-9b48-79896efc4207 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:52.068825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229793c-778c-4fec-9b48-79896efc4207 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:52.069097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229793c-778c-4fec-9b48-79896efc4207 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:52.072771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 130/249] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:52.077420 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229793c-778c-4fec-9b48-79896efc4207 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:52.110001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118e6a0-eab4-4f66-89e3-d1334e8f5b31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:52.111411 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118e6a0-eab4-4f66-89e3-d1334e8f5b31 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:52.134224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:52.197116 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229793c-778c-4fec-9b48-79896efc4207 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:52.229330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 120/250] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 20:59:52.230706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd57aa9-84f5-4791-ba02-3ac5a5eccc1a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:52.230998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd57aa9-84f5-4791-ba02-3ac5a5eccc1a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:52.231273 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd57aa9-84f5-4791-ba02-3ac5a5eccc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:52.239694 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f3b40-a849-499a-be80-2f579e522193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:52.240543 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f3b40-a849-499a-be80-2f579e522193 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:52.240992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f3b40-a849-499a-be80-2f579e522193 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:52.241398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f3b40-a849-499a-be80-2f579e522193 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:52.246150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f3b40-a849-499a-be80-2f579e522193 None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:52.250356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f3b40-a849-499a-be80-2f579e522193 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:52.253245 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 131/251] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:52 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:52.254220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f3b40-a849-499a-be80-2f579e522193 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:52.277695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2dd57aa9-84f5-4791-ba02-3ac5a5eccc1a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:52.278950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2dd57aa9-84f5-4791-ba02-3ac5a5eccc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:52.313180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:52.368855 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f3b40-a849-499a-be80-2f579e522193 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:52.418369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cccaf-77a6-4cb6-88aa-ae5298f69383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:52.419322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cccaf-77a6-4cb6-88aa-ae5298f69383 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:52.419767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cccaf-77a6-4cb6-88aa-ae5298f69383 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:52.420178 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cccaf-77a6-4cb6-88aa-ae5298f69383 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:52.421565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cccaf-77a6-4cb6-88aa-ae5298f69383 None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:52.423250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cccaf-77a6-4cb6-88aa-ae5298f69383 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:52.425444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 132/252] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:52 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:52.426420 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cccaf-77a6-4cb6-88aa-ae5298f69383 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:52.438564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 121/253] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:52.439508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd57aa9-84f5-4791-ba02-3ac5a5eccc1a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:52.533051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:52.559444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cccaf-77a6-4cb6-88aa-ae5298f69383 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:52.582816 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd57aa9-84f5-4791-ba02-3ac5a5eccc1a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:52.618103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:52.631737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ca6c1-f9d7-400a-88e3-4863138bc09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:52.632288 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ca6c1-f9d7-400a-88e3-4863138bc09b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:52.632568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ca6c1-f9d7-400a-88e3-4863138bc09b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:52.632866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ca6c1-f9d7-400a-88e3-4863138bc09b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:52.633606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ca6c1-f9d7-400a-88e3-4863138bc09b None admin] RBAC: Adding query filter params (role.id=6abadafbec7f43d2b2cc2df439c44eff, scope.project.id=402258177cb849268ec3155d32bedd84, user.id=c4a0d511d1bb4d66b636fe1651821f21) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:52.638087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ca6c1-f9d7-400a-88e3-4863138bc09b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:52.640335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ca6c1-f9d7-400a-88e3-4863138bc09b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:52.653049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 122/254] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:52 2022] GET /identity/v3/role_assignments?user.id=c4a0d511d1bb4d66b636fe1651821f21&scope.project.id=402258177cb849268ec3155d32bedd84&role.id=6abadafbec7f43d2b2cc2df439c44eff => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:52.654650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec57911-d6d8-432d-8ac5-b6e1bb152d50 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:52.654936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec57911-d6d8-432d-8ac5-b6e1bb152d50 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:52.655211 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec57911-d6d8-432d-8ac5-b6e1bb152d50 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:52.656698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 123/255] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:52.662560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196d2c8-0c9d-4298-92e1-ae9b5b41f888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:52.663275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d2c8-0c9d-4298-92e1-ae9b5b41f888 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:52.663555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d2c8-0c9d-4298-92e1-ae9b5b41f888 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:52.663829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d2c8-0c9d-4298-92e1-ae9b5b41f888 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84/users/252526882e334874a5173783d27d41c6/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:52.666717 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abc6da-7232-49f8-ad26-1ff5b3be8fd5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:52.666998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abc6da-7232-49f8-ad26-1ff5b3be8fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:52.667270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abc6da-7232-49f8-ad26-1ff5b3be8fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:52.668107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196d2c8-0c9d-4298-92e1-ae9b5b41f888 None admin] RBAC: Authorizing `identity:create_grant(project_id=402258177cb849268ec3155d32bedd84, user_id=252526882e334874a5173783d27d41c6, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:52.669675 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196d2c8-0c9d-4298-92e1-ae9b5b41f888 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:52.694323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 133/256] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 20:59:52 2022] PUT /identity/v3/projects/402258177cb849268ec3155d32bedd84/users/252526882e334874a5173783d27d41c6/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:59:52.714192 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-40abc6da-7232-49f8-ad26-1ff5b3be8fd5 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:52.715445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-40abc6da-7232-49f8-ad26-1ff5b3be8fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:52.822011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 124/257] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:52.936849 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196d2c8-0c9d-4298-92e1-ae9b5b41f888 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:53.086036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196d2c8-0c9d-4298-92e1-ae9b5b41f888 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:53.133437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeba17f-7aa0-4c33-9e53-bfa6da9ef3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:53.134021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeba17f-7aa0-4c33-9e53-bfa6da9ef3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:53.134384 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeba17f-7aa0-4c33-9e53-bfa6da9ef3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:53.134718 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeba17f-7aa0-4c33-9e53-bfa6da9ef3f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:53.151268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeba17f-7aa0-4c33-9e53-bfa6da9ef3f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-3eeba17f-7aa0-4c33-9e53-bfa6da9ef3f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:53.156841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:53.157700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:53.158489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:53.158489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:53.158489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:53.158489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:53.158489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:53.158489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:53.158489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:53.158489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:53.158489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:53.158489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:53.158489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:59:53.158489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:53.158489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 134/258] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 20:59:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:53.163945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abc6da-7232-49f8-ad26-1ff5b3be8fd5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:53.204199 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeba17f-7aa0-4c33-9e53-bfa6da9ef3f8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:53.214184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:53.277017 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:53.286051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abc6da-7232-49f8-ad26-1ff5b3be8fd5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:53.350552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fe227-3075-45c4-8ef5-cc654bab09a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:53.351182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fe227-3075-45c4-8ef5-cc654bab09a9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:53.351460 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fe227-3075-45c4-8ef5-cc654bab09a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:53.351759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fe227-3075-45c4-8ef5-cc654bab09a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:53.352466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fe227-3075-45c4-8ef5-cc654bab09a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:53.352797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fe227-3075-45c4-8ef5-cc654bab09a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:53.356888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fe227-3075-45c4-8ef5-cc654bab09a9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:53.377726 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 125/259] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:53.383685 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fe227-3075-45c4-8ef5-cc654bab09a9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:53.400158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeba17f-7aa0-4c33-9e53-bfa6da9ef3f8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:53.437863 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06241a2-24c1-4c22-9f81-3098e679c515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:53.438879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06241a2-24c1-4c22-9f81-3098e679c515 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:53.439301 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06241a2-24c1-4c22-9f81-3098e679c515 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:53.439576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06241a2-24c1-4c22-9f81-3098e679c515 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:53.453324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06241a2-24c1-4c22-9f81-3098e679c515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e06241a2-24c1-4c22-9f81-3098e679c515 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:53.458731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:53.459612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:53.460423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:53.460423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:53.460423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:53.460423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:53.460423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:53.460423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:53.460423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:53.460423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:53.460423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:53.460423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:53.460423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 20:59:53.460423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:53.460423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 135/260] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 20:59:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:59:53.469027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06241a2-24c1-4c22-9f81-3098e679c515 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:53.588427 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06241a2-24c1-4c22-9f81-3098e679c515 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:53.593372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:53.608058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fe227-3075-45c4-8ef5-cc654bab09a9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:53.610627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:53.630141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0f918-6194-4e9c-ac86-6d4ae5474c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:53.631101 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0f918-6194-4e9c-ac86-6d4ae5474c9c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:53.631519 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0f918-6194-4e9c-ac86-6d4ae5474c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:53.632025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0f918-6194-4e9c-ac86-6d4ae5474c9c None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:53.632878 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0f918-6194-4e9c-ac86-6d4ae5474c9c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:53.633432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0f918-6194-4e9c-ac86-6d4ae5474c9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:53.635234 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0f918-6194-4e9c-ac86-6d4ae5474c9c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:53.645336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 136/261] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 20:59:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:53.651321 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0f918-6194-4e9c-ac86-6d4ae5474c9c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:53.681382 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6d5dc-43e3-4fc9-b44a-5b77d38fd1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:53.682048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d5dc-43e3-4fc9-b44a-5b77d38fd1ee None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:53.682329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d5dc-43e3-4fc9-b44a-5b77d38fd1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:53.682601 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d5dc-43e3-4fc9-b44a-5b77d38fd1ee None admin] PATH_INFO: `/v3/users/3917da18a49f40c2be38ac6025b032b6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:53.683822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6d5dc-43e3-4fc9-b44a-5b77d38fd1ee None admin] RBAC: Authorizing `identity:get_user(user_id=3917da18a49f40c2be38ac6025b032b6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:53.685421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6d5dc-43e3-4fc9-b44a-5b77d38fd1ee None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:53.692391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 126/262] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:53 2022] GET /identity/v3/users/3917da18a49f40c2be38ac6025b032b6 => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:53.693883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6d5dc-43e3-4fc9-b44a-5b77d38fd1ee None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:53.763962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0f918-6194-4e9c-ac86-6d4ae5474c9c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:53.801884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f388124-73d1-4cff-84df-9b50c2ea2db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:53.802826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388124-73d1-4cff-84df-9b50c2ea2db8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:53.803274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388124-73d1-4cff-84df-9b50c2ea2db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:53.803686 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388124-73d1-4cff-84df-9b50c2ea2db8 None admin] PATH_INFO: `/v3/users/3917da18a49f40c2be38ac6025b032b6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:53.805125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f388124-73d1-4cff-84df-9b50c2ea2db8 None admin] RBAC: Authorizing `identity:get_user(user_id=3917da18a49f40c2be38ac6025b032b6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:53.806869 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f388124-73d1-4cff-84df-9b50c2ea2db8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:53.808687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 137/263] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:53 2022] GET /identity/v3/users/3917da18a49f40c2be38ac6025b032b6 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:53.814848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f388124-73d1-4cff-84df-9b50c2ea2db8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:53.853851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:53.878679 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6d5dc-43e3-4fc9-b44a-5b77d38fd1ee None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:53.969956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f2297-c296-429c-b7b1-70eb2d226677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:53.970593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f2297-c296-429c-b7b1-70eb2d226677 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:53.970872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f2297-c296-429c-b7b1-70eb2d226677 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:53.971152 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f2297-c296-429c-b7b1-70eb2d226677 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:54.011915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f388124-73d1-4cff-84df-9b50c2ea2db8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:54.075338 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a0134-a935-487a-b6cb-6d14d6294e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:54.075914 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a0134-a935-487a-b6cb-6d14d6294e96 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:54.076191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a0134-a935-487a-b6cb-6d14d6294e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:54.076465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a0134-a935-487a-b6cb-6d14d6294e96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:54.093381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:54.173757 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f2297-c296-429c-b7b1-70eb2d226677 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:54.175267 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f2297-c296-429c-b7b1-70eb2d226677 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:54.224199 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a0134-a935-487a-b6cb-6d14d6294e96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:54.225712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a0134-a935-487a-b6cb-6d14d6294e96 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:54.286526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:54.382618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 138/264] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 20:59:54.389544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a0134-a935-487a-b6cb-6d14d6294e96 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:54.501635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a0134-a935-487a-b6cb-6d14d6294e96 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:54.539233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76aa963-3946-40ce-ba0e-0387de129e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:54.540074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76aa963-3946-40ce-ba0e-0387de129e3f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:54.540455 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76aa963-3946-40ce-ba0e-0387de129e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:54.540859 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76aa963-3946-40ce-ba0e-0387de129e3f None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:54.542290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76aa963-3946-40ce-ba0e-0387de129e3f None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:54.543508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 127/265] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 850 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 20:59:54.543953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76aa963-3946-40ce-ba0e-0387de129e3f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:54.546094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 139/266] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:54 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:54.549005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:54.549696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f2297-c296-429c-b7b1-70eb2d226677 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:54.553087 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76aa963-3946-40ce-ba0e-0387de129e3f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:54.790094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:54.821049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f2297-c296-429c-b7b1-70eb2d226677 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:54.842760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76aa963-3946-40ce-ba0e-0387de129e3f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:54.861231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca299d3-8b3f-4558-9909-0f62407db20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:54.862140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca299d3-8b3f-4558-9909-0f62407db20a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:54.862419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca299d3-8b3f-4558-9909-0f62407db20a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:54.862688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca299d3-8b3f-4558-9909-0f62407db20a None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:54.864574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca299d3-8b3f-4558-9909-0f62407db20a None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:54.866697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca299d3-8b3f-4558-9909-0f62407db20a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:54.868775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 128/267] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:54 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:54.869972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23005e6-dd8c-45b3-8af0-7a02cb7ec095 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:54.870684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23005e6-dd8c-45b3-8af0-7a02cb7ec095 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:54.870977 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23005e6-dd8c-45b3-8af0-7a02cb7ec095 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:54.872620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 129/268] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:54.874067 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac16f32-9b1b-40fb-875b-ce658ef88ee6 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:54.874693 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac16f32-9b1b-40fb-875b-ce658ef88ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:54.874968 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac16f32-9b1b-40fb-875b-ce658ef88ee6 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:54.877029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 130/269] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:54.880636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db0ae6-10eb-492a-be5a-cf4c31ee2fa5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:54.881225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db0ae6-10eb-492a-be5a-cf4c31ee2fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:54.881853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db0ae6-10eb-492a-be5a-cf4c31ee2fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:54.895520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b462851-7cc8-4679-a2cb-d57953084144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:54.896678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b462851-7cc8-4679-a2cb-d57953084144 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:54.896963 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b462851-7cc8-4679-a2cb-d57953084144 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:54.897235 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b462851-7cc8-4679-a2cb-d57953084144 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:54.898112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b462851-7cc8-4679-a2cb-d57953084144 None admin] RBAC: Adding query filter params (role.id=e945f087a5bc4c38bd8848782f76c214, scope.project.id=402258177cb849268ec3155d32bedd84, user.id=3917da18a49f40c2be38ac6025b032b6) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:54.898562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b462851-7cc8-4679-a2cb-d57953084144 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:54.900235 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b462851-7cc8-4679-a2cb-d57953084144 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:54.903138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e8db0ae6-10eb-492a-be5a-cf4c31ee2fa5 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:54.904627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e8db0ae6-10eb-492a-be5a-cf4c31ee2fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:54.907340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 140/270] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:54 2022] GET /identity/v3/role_assignments?user.id=3917da18a49f40c2be38ac6025b032b6&scope.project.id=402258177cb849268ec3155d32bedd84&role.id=e945f087a5bc4c38bd8848782f76c214 => generated 608 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:54.909472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b462851-7cc8-4679-a2cb-d57953084144 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:55.007055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 131/271] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:55.008395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479bbf7-ff1c-46c6-8b76-bea6c167b6d6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:55.008696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479bbf7-ff1c-46c6-8b76-bea6c167b6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:55.008976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479bbf7-ff1c-46c6-8b76-bea6c167b6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:55.030045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:55.043149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9479bbf7-ff1c-46c6-8b76-bea6c167b6d6 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:55.044343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9479bbf7-ff1c-46c6-8b76-bea6c167b6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:55.079142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b462851-7cc8-4679-a2cb-d57953084144 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:55.119630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df7a35-6443-4be7-bfed-4cb9f438093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:55.120585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df7a35-6443-4be7-bfed-4cb9f438093a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:55.121037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df7a35-6443-4be7-bfed-4cb9f438093a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:55.121450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df7a35-6443-4be7-bfed-4cb9f438093a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:55.122454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df7a35-6443-4be7-bfed-4cb9f438093a None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=402258177cb849268ec3155d32bedd84, user.id=3917da18a49f40c2be38ac6025b032b6) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:55.122933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df7a35-6443-4be7-bfed-4cb9f438093a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:55.124587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df7a35-6443-4be7-bfed-4cb9f438093a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:55.131567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 141/272] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:54 2022] GET /identity/v3/role_assignments?user.id=3917da18a49f40c2be38ac6025b032b6&scope.project.id=402258177cb849268ec3155d32bedd84&role.id=3b30504991194002b41f8587d0851c10 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:55.169084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:55.174662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 132/273] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 20:59:55.458201 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df7a35-6443-4be7-bfed-4cb9f438093a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:55.533858 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479bbf7-ff1c-46c6-8b76-bea6c167b6d6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:55.583745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df7a35-6443-4be7-bfed-4cb9f438093a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:55.610533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:55.625680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ac204-eb63-42f0-b3ce-a9bedb4476ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:55.627281 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ac204-eb63-42f0-b3ce-a9bedb4476ad None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:55.627281 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ac204-eb63-42f0-b3ce-a9bedb4476ad None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:55.627281 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ac204-eb63-42f0-b3ce-a9bedb4476ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:55.628557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ac204-eb63-42f0-b3ce-a9bedb4476ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:55.630090 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ac204-eb63-42f0-b3ce-a9bedb4476ad None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:55.632055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 142/274] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:55.640904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ac204-eb63-42f0-b3ce-a9bedb4476ad None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:55.664995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479bbf7-ff1c-46c6-8b76-bea6c167b6d6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:55.702435 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d67a2-089a-4d8f-936a-ccafa86326c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:55.703412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d67a2-089a-4d8f-936a-ccafa86326c0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:55.703878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d67a2-089a-4d8f-936a-ccafa86326c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:55.704436 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d67a2-089a-4d8f-936a-ccafa86326c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:55.716623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d67a2-089a-4d8f-936a-ccafa86326c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-998d67a2-089a-4d8f-936a-ccafa86326c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:55.722092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:55.723044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:55.723808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:55.723808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:55.723808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:55.723808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:55.723808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:55.723808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:55.723808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:55.723808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:55.723808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:55.723808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:55.723808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:55.723808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:55.726184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 133/275] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:59:55.732402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d67a2-089a-4d8f-936a-ccafa86326c0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:55.758085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ac204-eb63-42f0-b3ce-a9bedb4476ad None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:55.798714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270510b7-10ba-4266-8847-1a1ff5ae6ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:55.799339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270510b7-10ba-4266-8847-1a1ff5ae6ef9 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:55.799617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270510b7-10ba-4266-8847-1a1ff5ae6ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:55.799890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270510b7-10ba-4266-8847-1a1ff5ae6ef9 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:55.800636 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270510b7-10ba-4266-8847-1a1ff5ae6ef9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:55.801657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:55.803490 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270510b7-10ba-4266-8847-1a1ff5ae6ef9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:55.830823 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 143/276] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 20:59:55 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:59:55.859361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d67a2-089a-4d8f-936a-ccafa86326c0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:55.866936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aad082-367c-49d4-9510-107916503100 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:55.867452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aad082-367c-49d4-9510-107916503100 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:55.867862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aad082-367c-49d4-9510-107916503100 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:55.870226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 144/277] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:59:55.876222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a2d63-9d90-4a4b-8a7b-474e7730bdae None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:55.876502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a2d63-9d90-4a4b-8a7b-474e7730bdae None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:55.877004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a2d63-9d90-4a4b-8a7b-474e7730bdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:55.898057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2f4a2d63-9d90-4a4b-8a7b-474e7730bdae None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:55.899504 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2f4a2d63-9d90-4a4b-8a7b-474e7730bdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:55.908337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909d895-a25b-4f1e-b5fb-2198f38c5e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:55.908337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909d895-a25b-4f1e-b5fb-2198f38c5e0a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:55.908962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909d895-a25b-4f1e-b5fb-2198f38c5e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:55.908962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909d895-a25b-4f1e-b5fb-2198f38c5e0a None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:55.909602 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909d895-a25b-4f1e-b5fb-2198f38c5e0a None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:55.909954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909d895-a25b-4f1e-b5fb-2198f38c5e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:55.911412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909d895-a25b-4f1e-b5fb-2198f38c5e0a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:55.922872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 134/278] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:55.929005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909d895-a25b-4f1e-b5fb-2198f38c5e0a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:55.973175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:55.993213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 145/279] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:56.063531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909d895-a25b-4f1e-b5fb-2198f38c5e0a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:56.089041 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:56.101300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33ede3-9e43-42a0-bbd6-09d8f8916f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:56.101949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33ede3-9e43-42a0-bbd6-09d8f8916f07 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:56.102227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33ede3-9e43-42a0-bbd6-09d8f8916f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:56.102506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33ede3-9e43-42a0-bbd6-09d8f8916f07 None admin] PATH_INFO: `/v3/users/cb533823e6e74886979475f9399129e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:56.120471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15182a-48d0-40f2-8260-9038f92792d3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:56.120851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15182a-48d0-40f2-8260-9038f92792d3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:56.121143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15182a-48d0-40f2-8260-9038f92792d3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:56.122630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 146/280] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:59:56.123025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33ede3-9e43-42a0-bbd6-09d8f8916f07 None admin] RBAC: Authorizing `identity:get_user(user_id=cb533823e6e74886979475f9399129e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:56.124818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33ede3-9e43-42a0-bbd6-09d8f8916f07 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:56.126862 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 135/281] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:55 2022] GET /identity/v3/users/cb533823e6e74886979475f9399129e4 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:56.127959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4b36f-e7dd-42ca-a11c-d853cd4b5083 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:56.128198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4b36f-e7dd-42ca-a11c-d853cd4b5083 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:56.128472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4b36f-e7dd-42ca-a11c-d853cd4b5083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:56.133253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33ede3-9e43-42a0-bbd6-09d8f8916f07 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:56.149721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bdf4b36f-e7dd-42ca-a11c-d853cd4b5083 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:56.150959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bdf4b36f-e7dd-42ca-a11c-d853cd4b5083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:56.227348 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 147/282] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:56.250638 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33ede3-9e43-42a0-bbd6-09d8f8916f07 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:56.287395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee9442-cdf1-4cf5-a4ab-2c9e994e6253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:56.288021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee9442-cdf1-4cf5-a4ab-2c9e994e6253 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:56.288316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee9442-cdf1-4cf5-a4ab-2c9e994e6253 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:56.288571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee9442-cdf1-4cf5-a4ab-2c9e994e6253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:56.289340 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee9442-cdf1-4cf5-a4ab-2c9e994e6253 None admin] RBAC: Adding query filter params (role.id=180edbb8a9e74555af89d6a7639f819a, user.id=cb533823e6e74886979475f9399129e4, scope.system=all) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:56.289645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee9442-cdf1-4cf5-a4ab-2c9e994e6253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:56.291154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4b36f-e7dd-42ca-a11c-d853cd4b5083 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:56.291345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee9442-cdf1-4cf5-a4ab-2c9e994e6253 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:56.301643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 136/283] 10.222.0.79 () {56 vars in 1308 bytes} [Tue Sep 6 20:59:56 2022] GET /identity/v3/role_assignments?user.id=cb533823e6e74886979475f9399129e4&scope.system=all&role.id=180edbb8a9e74555af89d6a7639f819a => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:56.421084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4b36f-e7dd-42ca-a11c-d853cd4b5083 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:56.434171 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:56.461842 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced897b-1a4d-4d5f-83db-2596b1d10eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:56.462631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced897b-1a4d-4d5f-83db-2596b1d10eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:56.462913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced897b-1a4d-4d5f-83db-2596b1d10eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:56.463185 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced897b-1a4d-4d5f-83db-2596b1d10eb6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:56.475371 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced897b-1a4d-4d5f-83db-2596b1d10eb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-cced897b-1a4d-4d5f-83db-2596b1d10eb6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:56.480937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:56.481870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:56.482647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:56.482647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:56.482647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:56.482647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:56.482647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:56.482647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:56.482647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:56.482647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:56.482647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:56.482647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:56.482647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 20:59:56.482647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:56.482647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 148/284] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:56.489803 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee9442-cdf1-4cf5-a4ab-2c9e994e6253 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:56.513083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced897b-1a4d-4d5f-83db-2596b1d10eb6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:56.554893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:56.663291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee9442-cdf1-4cf5-a4ab-2c9e994e6253 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:56.668162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced897b-1a4d-4d5f-83db-2596b1d10eb6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:56.707082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82214ee0-8bf0-4fa7-9311-3c977d54c6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:56.708018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82214ee0-8bf0-4fa7-9311-3c977d54c6ac None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:56.708297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82214ee0-8bf0-4fa7-9311-3c977d54c6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:56.708565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82214ee0-8bf0-4fa7-9311-3c977d54c6ac None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:56.709605 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82214ee0-8bf0-4fa7-9311-3c977d54c6ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:56.709909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82214ee0-8bf0-4fa7-9311-3c977d54c6ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:56.711342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82214ee0-8bf0-4fa7-9311-3c977d54c6ac None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:56.721561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 137/285] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:56.727277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82214ee0-8bf0-4fa7-9311-3c977d54c6ac None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:56.754133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91678f9-7a8e-47ec-bdd0-bd9c390b63ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:56.754856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91678f9-7a8e-47ec-bdd0-bd9c390b63ce None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:56.755133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91678f9-7a8e-47ec-bdd0-bd9c390b63ce None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:56.755401 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91678f9-7a8e-47ec-bdd0-bd9c390b63ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:56.770227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91678f9-7a8e-47ec-bdd0-bd9c390b63ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f91678f9-7a8e-47ec-bdd0-bd9c390b63ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:56.776108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:56.777076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:56.777872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:56.777872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:56.777872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:56.777872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:56.777872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:56.777872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:56.777872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:56.777872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:56.777872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:56.777872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:56.777872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:56.777872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 20:59:56.778971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 149/286] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:59:56.785376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91678f9-7a8e-47ec-bdd0-bd9c390b63ce None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:56.839572 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:56.898830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82214ee0-8bf0-4fa7-9311-3c977d54c6ac None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:56.906500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:56.941529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce27449-c0d0-4b09-91ab-5bac3d354baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:56.942173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce27449-c0d0-4b09-91ab-5bac3d354baf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:56.942451 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce27449-c0d0-4b09-91ab-5bac3d354baf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:56.942725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce27449-c0d0-4b09-91ab-5bac3d354baf None admin] PATH_INFO: `/v3/users/c4a0d511d1bb4d66b636fe1651821f21` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:56.944347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce27449-c0d0-4b09-91ab-5bac3d354baf None admin] RBAC: Authorizing `identity:get_user(user_id=c4a0d511d1bb4d66b636fe1651821f21)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:56.945895 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce27449-c0d0-4b09-91ab-5bac3d354baf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:56.952686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 138/287] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:56 2022] GET /identity/v3/users/c4a0d511d1bb4d66b636fe1651821f21 => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:56.954979 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce27449-c0d0-4b09-91ab-5bac3d354baf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:56.975854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91678f9-7a8e-47ec-bdd0-bd9c390b63ce None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:57.048329 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46133836-3e6b-4d22-bc16-cfba8d62f024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:57.048882 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46133836-3e6b-4d22-bc16-cfba8d62f024 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:57.049165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46133836-3e6b-4d22-bc16-cfba8d62f024 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:57.049434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46133836-3e6b-4d22-bc16-cfba8d62f024 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:57.050108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46133836-3e6b-4d22-bc16-cfba8d62f024 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:57.050411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46133836-3e6b-4d22-bc16-cfba8d62f024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:57.051812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46133836-3e6b-4d22-bc16-cfba8d62f024 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:57.062152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 150/288] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:57.074039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46133836-3e6b-4d22-bc16-cfba8d62f024 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:57.118459 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce27449-c0d0-4b09-91ab-5bac3d354baf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:57.166013 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbea87e-3758-4b2a-9478-061671485f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:57.168821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbea87e-3758-4b2a-9478-061671485f05 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:57.169619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbea87e-3758-4b2a-9478-061671485f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:57.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbea87e-3758-4b2a-9478-061671485f05 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:57.172569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbea87e-3758-4b2a-9478-061671485f05 None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:57.175731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbea87e-3758-4b2a-9478-061671485f05 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:57.177904 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 139/289] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:56 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:57.184148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbea87e-3758-4b2a-9478-061671485f05 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:57.269379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:57.283448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:57.295059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46133836-3e6b-4d22-bc16-cfba8d62f024 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:57.347052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbea87e-3758-4b2a-9478-061671485f05 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:57.367507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa17e5-a717-4be3-947b-af48b8ea031a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:57.368175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa17e5-a717-4be3-947b-af48b8ea031a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:57.368476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa17e5-a717-4be3-947b-af48b8ea031a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:57.368743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa17e5-a717-4be3-947b-af48b8ea031a None admin] PATH_INFO: `/v3/users/252526882e334874a5173783d27d41c6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:57.369990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa17e5-a717-4be3-947b-af48b8ea031a None admin] RBAC: Authorizing `identity:get_user(user_id=252526882e334874a5173783d27d41c6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:57.371639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa17e5-a717-4be3-947b-af48b8ea031a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:57.376639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 151/290] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:57 2022] GET /identity/v3/users/252526882e334874a5173783d27d41c6 => generated 297 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:57.384533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa17e5-a717-4be3-947b-af48b8ea031a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:57.396179 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1076f9-0ec9-4947-9c76-9503feeba0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:57.396885 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1076f9-0ec9-4947-9c76-9503feeba0a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:57.397161 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1076f9-0ec9-4947-9c76-9503feeba0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:57.397431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1076f9-0ec9-4947-9c76-9503feeba0a2 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84/users/c4a0d511d1bb4d66b636fe1651821f21/roles/6abadafbec7f43d2b2cc2df439c44eff` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:57.399347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1076f9-0ec9-4947-9c76-9503feeba0a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=402258177cb849268ec3155d32bedd84, user_id=c4a0d511d1bb4d66b636fe1651821f21, role_id=6abadafbec7f43d2b2cc2df439c44eff)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:57.403833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1076f9-0ec9-4947-9c76-9503feeba0a2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:57.418485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 140/291] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 20:59:57 2022] PUT /identity/v3/projects/402258177cb849268ec3155d32bedd84/users/c4a0d511d1bb4d66b636fe1651821f21/roles/6abadafbec7f43d2b2cc2df439c44eff => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:59:57.522374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa17e5-a717-4be3-947b-af48b8ea031a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:57.561971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536869b-19d6-407a-8822-57ce9ed8596e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:57.562599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536869b-19d6-407a-8822-57ce9ed8596e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:57.562877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536869b-19d6-407a-8822-57ce9ed8596e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:57.563214 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536869b-19d6-407a-8822-57ce9ed8596e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:57.593856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:57.641663 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536869b-19d6-407a-8822-57ce9ed8596e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:57.643213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536869b-19d6-407a-8822-57ce9ed8596e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:57.755406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 152/292] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 20:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 20:59:57.761272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1076f9-0ec9-4947-9c76-9503feeba0a2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:57.874160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1076f9-0ec9-4947-9c76-9503feeba0a2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:57.875897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:57.912916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad884cbe-b3e5-4f64-b96b-4f3bd0a7bccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:57.913565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad884cbe-b3e5-4f64-b96b-4f3bd0a7bccc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:57.913840 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad884cbe-b3e5-4f64-b96b-4f3bd0a7bccc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:57.914584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad884cbe-b3e5-4f64-b96b-4f3bd0a7bccc None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:57.915738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad884cbe-b3e5-4f64-b96b-4f3bd0a7bccc None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:57.917363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad884cbe-b3e5-4f64-b96b-4f3bd0a7bccc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:57.919352 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 141/293] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 20:59:57 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:57.925249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536869b-19d6-407a-8822-57ce9ed8596e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:57.997769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:58.043399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536869b-19d6-407a-8822-57ce9ed8596e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:58.079061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac130b57-260d-4acf-93c5-0988278a83ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:58.079727 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac130b57-260d-4acf-93c5-0988278a83ae None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:58.080066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac130b57-260d-4acf-93c5-0988278a83ae None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:58.080385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac130b57-260d-4acf-93c5-0988278a83ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:58.081162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac130b57-260d-4acf-93c5-0988278a83ae None admin] RBAC: Adding query filter params (role.id=180edbb8a9e74555af89d6a7639f819a, scope.project.id=402258177cb849268ec3155d32bedd84, user.id=252526882e334874a5173783d27d41c6) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:58.081465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac130b57-260d-4acf-93c5-0988278a83ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:58.082931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac130b57-260d-4acf-93c5-0988278a83ae None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:58.089678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 153/294] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 20:59:57 2022] GET /identity/v3/role_assignments?user.id=252526882e334874a5173783d27d41c6&scope.project.id=402258177cb849268ec3155d32bedd84&role.id=180edbb8a9e74555af89d6a7639f819a => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:58.802739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c626df-b0f2-44a1-b561-2a6e6911575d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:58.803298 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c626df-b0f2-44a1-b561-2a6e6911575d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:58.803372 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c626df-b0f2-44a1-b561-2a6e6911575d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:58.804853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 142/295] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:58.811670 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995ad89-0f7c-4cd0-82e6-8ae2fefef400 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:58.812499 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995ad89-0f7c-4cd0-82e6-8ae2fefef400 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:58.812939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995ad89-0f7c-4cd0-82e6-8ae2fefef400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:58.838059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0995ad89-0f7c-4cd0-82e6-8ae2fefef400 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:58.839590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0995ad89-0f7c-4cd0-82e6-8ae2fefef400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:58.849005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a17c1-44c3-4b29-b33e-94c19450a203 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:58.849005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a17c1-44c3-4b29-b33e-94c19450a203 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:58.849268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a17c1-44c3-4b29-b33e-94c19450a203 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:58.850782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 143/296] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:58.856234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc954418-6bfb-46f7-a6d1-4cbf71269c41 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:58.856508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc954418-6bfb-46f7-a6d1-4cbf71269c41 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:58.856905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc954418-6bfb-46f7-a6d1-4cbf71269c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:58.877553 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cc954418-6bfb-46f7-a6d1-4cbf71269c41 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:58.878817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cc954418-6bfb-46f7-a6d1-4cbf71269c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:58.921224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 154/297] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:58.922470 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a12a90-a109-457b-b6db-6b46dc39a35f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:58.922756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a12a90-a109-457b-b6db-6b46dc39a35f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:58.923027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a12a90-a109-457b-b6db-6b46dc39a35f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:58.924400 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 155/298] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 20:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:59:58.929645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a7658-50ec-4a3a-9c79-c4a2830c89a5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:58.929922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a7658-50ec-4a3a-9c79-c4a2830c89a5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:58.930338 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a7658-50ec-4a3a-9c79-c4a2830c89a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:58.951856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-712a7658-50ec-4a3a-9c79-c4a2830c89a5 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:59:58.953391 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-712a7658-50ec-4a3a-9c79-c4a2830c89a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:59:58.976811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 144/299] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:59.020273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:59.042471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 156/300] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 20:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 20:59:59.186330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc954418-6bfb-46f7-a6d1-4cbf71269c41 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:59.255301 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a7658-50ec-4a3a-9c79-c4a2830c89a5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:59.303511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:59.308690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc954418-6bfb-46f7-a6d1-4cbf71269c41 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:59.354875 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f081f-c9f4-461d-ba2f-0a2cd7130369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:59.355770 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f081f-c9f4-461d-ba2f-0a2cd7130369 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:59.356441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f081f-c9f4-461d-ba2f-0a2cd7130369 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:59.356796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f081f-c9f4-461d-ba2f-0a2cd7130369 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:59.362049 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a7658-50ec-4a3a-9c79-c4a2830c89a5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:59.372991 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f081f-c9f4-461d-ba2f-0a2cd7130369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3a6f081f-c9f4-461d-ba2f-0a2cd7130369 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:59.377912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:59.378747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:59.379511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:59.379511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:59.379511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:59.379511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:59.379511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:59.379511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:59.379511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:59.379511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:59.379511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:59.379511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:59.379511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 20:59:59.379511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:59.379511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 145/301] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:59.380134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f081f-c9f4-461d-ba2f-0a2cd7130369 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:59.408905 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e33286-5cb9-45e8-97ec-bf89484219d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:59.409819 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e33286-5cb9-45e8-97ec-bf89484219d4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:59.410086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e33286-5cb9-45e8-97ec-bf89484219d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:59.410352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e33286-5cb9-45e8-97ec-bf89484219d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:59.411464 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e33286-5cb9-45e8-97ec-bf89484219d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:59.412989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e33286-5cb9-45e8-97ec-bf89484219d4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:59.416897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 157/302] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 20:59:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:59:59.416897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e33286-5cb9-45e8-97ec-bf89484219d4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:59.474458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:59.500945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f081f-c9f4-461d-ba2f-0a2cd7130369 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:59.536932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e33286-5cb9-45e8-97ec-bf89484219d4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:59.539206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6180608-f585-4fb2-b55f-5248cc5a36d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:59.539987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6180608-f585-4fb2-b55f-5248cc5a36d0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:59.540258 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6180608-f585-4fb2-b55f-5248cc5a36d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:59.540574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6180608-f585-4fb2-b55f-5248cc5a36d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:59.552363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6180608-f585-4fb2-b55f-5248cc5a36d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b6180608-f585-4fb2-b55f-5248cc5a36d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:59:59.558883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:59:59.560009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 146/303] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 20:59:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:59:59.561548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6180608-f585-4fb2-b55f-5248cc5a36d0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:59.585494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ebed1-295c-4bd9-b474-b75088d9617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:59.586587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ebed1-295c-4bd9-b474-b75088d9617f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:59.587064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ebed1-295c-4bd9-b474-b75088d9617f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:59.590240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ebed1-295c-4bd9-b474-b75088d9617f None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:59.590656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ebed1-295c-4bd9-b474-b75088d9617f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:59.591032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ebed1-295c-4bd9-b474-b75088d9617f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:59.591190 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:59.592629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ebed1-295c-4bd9-b474-b75088d9617f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:59.607446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 158/304] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:59.607446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ebed1-295c-4bd9-b474-b75088d9617f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:59.697312 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6180608-f585-4fb2-b55f-5248cc5a36d0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:59.720131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ebed1-295c-4bd9-b474-b75088d9617f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:59.734629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:59.735900 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af079076-2615-4415-97a6-54fd6078ba18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:59.736577 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af079076-2615-4415-97a6-54fd6078ba18 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:59.736918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af079076-2615-4415-97a6-54fd6078ba18 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:59.737334 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af079076-2615-4415-97a6-54fd6078ba18 None admin] PATH_INFO: `/v3/groups` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:59.737980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af079076-2615-4415-97a6-54fd6078ba18 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:59.739556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af079076-2615-4415-97a6-54fd6078ba18 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:59.756796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 147/305] 10.222.0.79 () {60 vars in 1134 bytes} [Tue Sep 6 20:59:59 2022] POST /identity/v3/groups => generated 229 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:59:59.757967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af079076-2615-4415-97a6-54fd6078ba18 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:59.769469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfee4f1-2e98-46cd-a93b-e43887a5443b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:59.770216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfee4f1-2e98-46cd-a93b-e43887a5443b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:59.770494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfee4f1-2e98-46cd-a93b-e43887a5443b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:59.770762 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfee4f1-2e98-46cd-a93b-e43887a5443b None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:59.771455 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfee4f1-2e98-46cd-a93b-e43887a5443b None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:59:59.771753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfee4f1-2e98-46cd-a93b-e43887a5443b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:59.774274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfee4f1-2e98-46cd-a93b-e43887a5443b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:59.795355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 159/306] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 20:59:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:59.797248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfee4f1-2e98-46cd-a93b-e43887a5443b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:59.876957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:59:59.916996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af079076-2615-4415-97a6-54fd6078ba18 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:59.942146 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfee4f1-2e98-46cd-a93b-e43887a5443b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:59:59.957436 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb66a46-4855-4a11-968b-3fd1b1e60fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:59.958102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb66a46-4855-4a11-968b-3fd1b1e60fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:59.958381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb66a46-4855-4a11-968b-3fd1b1e60fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:59.958664 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb66a46-4855-4a11-968b-3fd1b1e60fa1 None admin] PATH_INFO: `/v3/users/c974cc998e1e4a2081569082d11ab599` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:59.978656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb66a46-4855-4a11-968b-3fd1b1e60fa1 None admin] RBAC: Authorizing `identity:get_user(user_id=c974cc998e1e4a2081569082d11ab599)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:59.980924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb66a46-4855-4a11-968b-3fd1b1e60fa1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:59.983739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 148/307] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:59 2022] GET /identity/v3/users/c974cc998e1e4a2081569082d11ab599 => generated 293 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:59:59.988161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34512633-88d1-4eb2-9045-dbd684c25ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:59:59.988848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34512633-88d1-4eb2-9045-dbd684c25ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:59:59.989188 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34512633-88d1-4eb2-9045-dbd684c25ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:59:59.989526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34512633-88d1-4eb2-9045-dbd684c25ab3 None admin] PATH_INFO: `/v3/users/cb533823e6e74886979475f9399129e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:59:59.990770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34512633-88d1-4eb2-9045-dbd684c25ab3 None admin] RBAC: Authorizing `identity:get_user(user_id=cb533823e6e74886979475f9399129e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:59:59.991498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb66a46-4855-4a11-968b-3fd1b1e60fa1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:59:59.992381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34512633-88d1-4eb2-9045-dbd684c25ab3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:59:59.998858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 160/308] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 20:59:59 2022] GET /identity/v3/users/cb533823e6e74886979475f9399129e4 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:00.002320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34512633-88d1-4eb2-9045-dbd684c25ab3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:00.070978 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:00.141681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb66a46-4855-4a11-968b-3fd1b1e60fa1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:00.146341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34512633-88d1-4eb2-9045-dbd684c25ab3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:00.181597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebf48d-8f15-4ab9-a612-d858b33b7bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:00.181812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514be11-5ff4-4c4b-94fd-6d2a122ff427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:00.182364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514be11-5ff4-4c4b-94fd-6d2a122ff427 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:00.182638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514be11-5ff4-4c4b-94fd-6d2a122ff427 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:00.182910 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514be11-5ff4-4c4b-94fd-6d2a122ff427 None admin] PATH_INFO: `/v3/system/users/cb533823e6e74886979475f9399129e4/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:00.183065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebf48d-8f15-4ab9-a612-d858b33b7bfc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:00.183490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebf48d-8f15-4ab9-a612-d858b33b7bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:00.184217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebf48d-8f15-4ab9-a612-d858b33b7bfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:00.184378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514be11-5ff4-4c4b-94fd-6d2a122ff427 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cb533823e6e74886979475f9399129e4, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:00.185432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebf48d-8f15-4ab9-a612-d858b33b7bfc None admin] RBAC: Adding query filter params (role.id=6abadafbec7f43d2b2cc2df439c44eff, user.id=c974cc998e1e4a2081569082d11ab599, scope.system=all) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:00.186249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514be11-5ff4-4c4b-94fd-6d2a122ff427 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:00.186355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebf48d-8f15-4ab9-a612-d858b33b7bfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:00.188038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebf48d-8f15-4ab9-a612-d858b33b7bfc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:00.194751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 149/309] 10.222.0.79 () {56 vars in 1308 bytes} [Tue Sep 6 20:59:59 2022] GET /identity/v3/role_assignments?user.id=c974cc998e1e4a2081569082d11ab599&scope.system=all&role.id=6abadafbec7f43d2b2cc2df439c44eff => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:00.204768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 161/310] 10.222.0.79 () {58 vars in 1334 bytes} [Tue Sep 6 21:00:00 2022] PUT /identity/v3/system/users/cb533823e6e74886979475f9399129e4/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:00:00.209332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e0cb3-63fa-4b91-b920-41509eced5eb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:00.210733 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e0cb3-63fa-4b91-b920-41509eced5eb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:00.211841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e0cb3-63fa-4b91-b920-41509eced5eb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:00.216237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 150/311] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:00.223397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2d9f4-d1a5-46ca-b911-02eeab45a37b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:00.223827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2d9f4-d1a5-46ca-b911-02eeab45a37b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:00.224176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2d9f4-d1a5-46ca-b911-02eeab45a37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:00.247573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-adb2d9f4-d1a5-46ca-b911-02eeab45a37b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:00.248997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-adb2d9f4-d1a5-46ca-b911-02eeab45a37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:00.349417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 162/312] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:00.640434 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e0cb3-63fa-4b91-b920-41509eced5eb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:00.643874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:00.755144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e0cb3-63fa-4b91-b920-41509eced5eb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:00.801668 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06f770-a8f0-4a3b-a211-998c142e893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:00.802334 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06f770-a8f0-4a3b-a211-998c142e893c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:00.802611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06f770-a8f0-4a3b-a211-998c142e893c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:00.802873 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06f770-a8f0-4a3b-a211-998c142e893c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:00.815407 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06f770-a8f0-4a3b-a211-998c142e893c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6e06f770-a8f0-4a3b-a211-998c142e893c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:00.820339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:00.821240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:00.822314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:00.822314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:00.822314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:00.822314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:00.822314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:00.822314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:00.822314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:00.822314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:00.822314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:00.822314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:00.822314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 21:00:00.822314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:00.822314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 151/313] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:00.830186 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2d9f4-d1a5-46ca-b911-02eeab45a37b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:00.869264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:00.938642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2d9f4-d1a5-46ca-b911-02eeab45a37b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:00.975982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021412f7-5c76-45a7-aabd-e86c4c2b5ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:00.976625 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021412f7-5c76-45a7-aabd-e86c4c2b5ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:00.976897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021412f7-5c76-45a7-aabd-e86c4c2b5ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:00.977162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021412f7-5c76-45a7-aabd-e86c4c2b5ab7 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:00.978121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021412f7-5c76-45a7-aabd-e86c4c2b5ab7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:00.978419 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021412f7-5c76-45a7-aabd-e86c4c2b5ab7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:00.979912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021412f7-5c76-45a7-aabd-e86c4c2b5ab7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:00.989708 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 163/314] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 21:00:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:00.996126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06f770-a8f0-4a3b-a211-998c142e893c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:01.075934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:01.123530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06f770-a8f0-4a3b-a211-998c142e893c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:01.173532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3866f-d11a-4b48-bcbd-fcb3a8f994dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:01.174176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3866f-d11a-4b48-bcbd-fcb3a8f994dd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:01.174449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3866f-d11a-4b48-bcbd-fcb3a8f994dd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:01.174719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3866f-d11a-4b48-bcbd-fcb3a8f994dd None admin] PATH_INFO: `/v3/users/c4a0d511d1bb4d66b636fe1651821f21` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:01.175859 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3866f-d11a-4b48-bcbd-fcb3a8f994dd None admin] RBAC: Authorizing `identity:get_user(user_id=c4a0d511d1bb4d66b636fe1651821f21)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:01.177373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3866f-d11a-4b48-bcbd-fcb3a8f994dd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:01.179325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 152/315] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:00:00 2022] GET /identity/v3/users/c4a0d511d1bb4d66b636fe1651821f21 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:01.185459 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021412f7-5c76-45a7-aabd-e86c4c2b5ab7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:01.289126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:01.293494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021412f7-5c76-45a7-aabd-e86c4c2b5ab7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:01.328461 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737face-6e6d-405d-871e-3762d8505877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:01.329093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737face-6e6d-405d-871e-3762d8505877 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:01.329365 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737face-6e6d-405d-871e-3762d8505877 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:01.329636 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737face-6e6d-405d-871e-3762d8505877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:01.399316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737face-6e6d-405d-871e-3762d8505877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:01.400734 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737face-6e6d-405d-871e-3762d8505877 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:01.504600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:01.513088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 164/316] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:01.517763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3866f-d11a-4b48-bcbd-fcb3a8f994dd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:01.632693 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3866f-d11a-4b48-bcbd-fcb3a8f994dd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:01.663941 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:01.670789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf04a1-874d-4210-a581-0a144b28d55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:01.671432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf04a1-874d-4210-a581-0a144b28d55c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:01.671701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf04a1-874d-4210-a581-0a144b28d55c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:01.672018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf04a1-874d-4210-a581-0a144b28d55c None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:01.673233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf04a1-874d-4210-a581-0a144b28d55c None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:01.674691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf04a1-874d-4210-a581-0a144b28d55c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:01.676677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 153/317] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:01 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:01.682991 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737face-6e6d-405d-871e-3762d8505877 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:01.787748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737face-6e6d-405d-871e-3762d8505877 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:01.823827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3ccf0-3446-47ee-93f6-93ed1aa4449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:01.824425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3ccf0-3446-47ee-93f6-93ed1aa4449c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:01.824718 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3ccf0-3446-47ee-93f6-93ed1aa4449c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:01.824983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3ccf0-3446-47ee-93f6-93ed1aa4449c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:01.825760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef3ccf0-3446-47ee-93f6-93ed1aa4449c None admin] RBAC: Adding query filter params (role.id=6abadafbec7f43d2b2cc2df439c44eff, scope.project.id=402258177cb849268ec3155d32bedd84, user.id=c4a0d511d1bb4d66b636fe1651821f21) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:01.826058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef3ccf0-3446-47ee-93f6-93ed1aa4449c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:01.827482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef3ccf0-3446-47ee-93f6-93ed1aa4449c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:01.833861 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 165/318] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:01 2022] GET /identity/v3/role_assignments?user.id=c4a0d511d1bb4d66b636fe1651821f21&scope.project.id=402258177cb849268ec3155d32bedd84&role.id=6abadafbec7f43d2b2cc2df439c44eff => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:02.151260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61235ff5-5ed3-47ee-b75c-91ca8da7a914 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:02.151830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61235ff5-5ed3-47ee-b75c-91ca8da7a914 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:02.151830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61235ff5-5ed3-47ee-b75c-91ca8da7a914 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:02.153498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 154/319] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:02.159138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0be06c-101b-44d8-b3fe-902bceebed75 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:02.159617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0be06c-101b-44d8-b3fe-902bceebed75 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:02.159713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0be06c-101b-44d8-b3fe-902bceebed75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:02.178429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4e0be06c-101b-44d8-b3fe-902bceebed75 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:02.179624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4e0be06c-101b-44d8-b3fe-902bceebed75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:02.251786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:02.263289 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 166/320] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:02.494379 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dfd44-ec20-4175-8864-f45b7bfd41da None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:02.494581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dfd44-ec20-4175-8864-f45b7bfd41da None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:02.494790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dfd44-ec20-4175-8864-f45b7bfd41da None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:02.496296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 155/321] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:02.502305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36f04c-ced7-4d33-a755-a0fbee8f1816 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:02.502586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36f04c-ced7-4d33-a755-a0fbee8f1816 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:02.502852 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36f04c-ced7-4d33-a755-a0fbee8f1816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:02.522951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ca36f04c-ced7-4d33-a755-a0fbee8f1816 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:02.524191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ca36f04c-ced7-4d33-a755-a0fbee8f1816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:02.526010 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dfd44-ec20-4175-8864-f45b7bfd41da None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:02.620484 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 167/322] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:02.649094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c99e5-79c4-4c5e-957e-bf070af2981b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:02.649860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c99e5-79c4-4c5e-957e-bf070af2981b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:02.650279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c99e5-79c4-4c5e-957e-bf070af2981b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:02.651789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 168/323] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:00:02.652553 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dfd44-ec20-4175-8864-f45b7bfd41da None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:02.657292 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c7e8e-e9f3-43f1-a07d-3de1e2fe9502 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:02.657829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c7e8e-e9f3-43f1-a07d-3de1e2fe9502 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:02.658336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c7e8e-e9f3-43f1-a07d-3de1e2fe9502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:02.678985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-821c7e8e-e9f3-43f1-a07d-3de1e2fe9502 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:02.680194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-821c7e8e-e9f3-43f1-a07d-3de1e2fe9502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:02.689703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a4f3f-1e1b-4c40-885e-15e83b1bea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:02.690357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a4f3f-1e1b-4c40-885e-15e83b1bea36 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:02.690636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a4f3f-1e1b-4c40-885e-15e83b1bea36 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:02.690904 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a4f3f-1e1b-4c40-885e-15e83b1bea36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:02.692037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a4f3f-1e1b-4c40-885e-15e83b1bea36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:02.693679 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a4f3f-1e1b-4c40-885e-15e83b1bea36 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:02.695624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 156/324] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:02.701872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a4f3f-1e1b-4c40-885e-15e83b1bea36 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:02.769952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 169/325] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 21:00:02.782230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:02.824179 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a4f3f-1e1b-4c40-885e-15e83b1bea36 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:02.861433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75201458-7f90-413a-b33e-d83d2ff16f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:02.862054 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75201458-7f90-413a-b33e-d83d2ff16f5b None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:02.862328 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75201458-7f90-413a-b33e-d83d2ff16f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:02.862594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75201458-7f90-413a-b33e-d83d2ff16f5b None admin] PATH_INFO: `/v3/groups` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:02.863231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75201458-7f90-413a-b33e-d83d2ff16f5b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:02.864722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75201458-7f90-413a-b33e-d83d2ff16f5b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:02.888691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 157/326] 10.222.0.79 () {60 vars in 1134 bytes} [Tue Sep 6 21:00:02 2022] POST /identity/v3/groups => generated 226 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:00:02.894878 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c7e8e-e9f3-43f1-a07d-3de1e2fe9502 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:02.957925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:03.023400 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c7e8e-e9f3-43f1-a07d-3de1e2fe9502 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:03.034663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75201458-7f90-413a-b33e-d83d2ff16f5b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:03.071411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970a630-3fc6-4486-9c45-5ed49313e2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:03.072204 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970a630-3fc6-4486-9c45-5ed49313e2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:03.072488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970a630-3fc6-4486-9c45-5ed49313e2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:03.072593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970a630-3fc6-4486-9c45-5ed49313e2f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:03.085858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970a630-3fc6-4486-9c45-5ed49313e2f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9970a630-3fc6-4486-9c45-5ed49313e2f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:03.090678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:03.091690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:03.092458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:03.092458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:03.092458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:03.092458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:03.092458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:03.092458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:03.092458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:03.092458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:03.092458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:03.092458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:03.092458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:03.092458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:03.092458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 170/327] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:03.099599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970a630-3fc6-4486-9c45-5ed49313e2f8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:03.180917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75201458-7f90-413a-b33e-d83d2ff16f5b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:03.220699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:03.233945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f63ab-f9fc-4a56-ac98-da84f8966935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:03.234567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f63ab-f9fc-4a56-ac98-da84f8966935 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:03.234847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f63ab-f9fc-4a56-ac98-da84f8966935 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:03.235106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f63ab-f9fc-4a56-ac98-da84f8966935 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:03.235297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970a630-3fc6-4486-9c45-5ed49313e2f8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:03.248595 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f63ab-f9fc-4a56-ac98-da84f8966935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:03.255732 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-288f63ab-f9fc-4a56-ac98-da84f8966935 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:03.255893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:03.256748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:03.257629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:03.257629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:03.257629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:03.257629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:03.257629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:03.257629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:03.257629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:03.257629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:03.257629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:03.257629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:03.257629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 21:00:03.257629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:03.257629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 158/328] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:00:03.264211 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f63ab-f9fc-4a56-ac98-da84f8966935 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:03.279031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c0646-3e9a-492b-bea9-17781d5788fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:03.279511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c0646-3e9a-492b-bea9-17781d5788fa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:03.279894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c0646-3e9a-492b-bea9-17781d5788fa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:03.280224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c0646-3e9a-492b-bea9-17781d5788fa None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:03.280909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c0646-3e9a-492b-bea9-17781d5788fa None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:03.281216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c0646-3e9a-492b-bea9-17781d5788fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:03.282862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c0646-3e9a-492b-bea9-17781d5788fa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:03.293349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 171/329] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 21:00:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:03.299611 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c0646-3e9a-492b-bea9-17781d5788fa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:03.413665 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f63ab-f9fc-4a56-ac98-da84f8966935 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:03.442407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c0646-3e9a-492b-bea9-17781d5788fa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:03.457517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44569a3a-6b0d-46df-81e6-b1c00f5e5bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:03.458216 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44569a3a-6b0d-46df-81e6-b1c00f5e5bdb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:03.458652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44569a3a-6b0d-46df-81e6-b1c00f5e5bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:03.458943 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44569a3a-6b0d-46df-81e6-b1c00f5e5bdb None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:03.460075 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44569a3a-6b0d-46df-81e6-b1c00f5e5bdb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:03.460383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44569a3a-6b0d-46df-81e6-b1c00f5e5bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:03.462227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44569a3a-6b0d-46df-81e6-b1c00f5e5bdb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:03.472691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 159/330] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 21:00:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:03.479703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44569a3a-6b0d-46df-81e6-b1c00f5e5bdb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:03.495980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ba485-8002-4de8-b28b-4b18803202d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:03.496828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ba485-8002-4de8-b28b-4b18803202d1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:03.497113 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ba485-8002-4de8-b28b-4b18803202d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:03.497503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ba485-8002-4de8-b28b-4b18803202d1 None admin] PATH_INFO: `/v3/users/cb533823e6e74886979475f9399129e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:03.499437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ba485-8002-4de8-b28b-4b18803202d1 None admin] RBAC: Authorizing `identity:get_user(user_id=cb533823e6e74886979475f9399129e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:03.501119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ba485-8002-4de8-b28b-4b18803202d1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:03.503404 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 172/331] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:00:03 2022] GET /identity/v3/users/cb533823e6e74886979475f9399129e4 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:03.511065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ba485-8002-4de8-b28b-4b18803202d1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:03.549090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:03.573976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:03.618186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44569a3a-6b0d-46df-81e6-b1c00f5e5bdb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:03.654160 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ba485-8002-4de8-b28b-4b18803202d1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:03.659647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed09365-1fa5-4c86-9408-d12cf738f082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:03.660442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed09365-1fa5-4c86-9408-d12cf738f082 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:03.660866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed09365-1fa5-4c86-9408-d12cf738f082 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:03.661234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed09365-1fa5-4c86-9408-d12cf738f082 None admin] PATH_INFO: `/v3/users/c974cc998e1e4a2081569082d11ab599` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:03.662553 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed09365-1fa5-4c86-9408-d12cf738f082 None admin] RBAC: Authorizing `identity:get_user(user_id=c974cc998e1e4a2081569082d11ab599)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:03.664270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed09365-1fa5-4c86-9408-d12cf738f082 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:03.666804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 160/332] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:00:03 2022] GET /identity/v3/users/c974cc998e1e4a2081569082d11ab599 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:03.675421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed09365-1fa5-4c86-9408-d12cf738f082 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:03.713799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bf9da-98b2-4859-b456-7d4f4dbcc4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:03.715573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bf9da-98b2-4859-b456-7d4f4dbcc4ef None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:03.716134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bf9da-98b2-4859-b456-7d4f4dbcc4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:03.716600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bf9da-98b2-4859-b456-7d4f4dbcc4ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:03.717823 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bf9da-98b2-4859-b456-7d4f4dbcc4ef None admin] RBAC: Adding query filter params (role.id=180edbb8a9e74555af89d6a7639f819a, user.id=cb533823e6e74886979475f9399129e4, scope.system=all) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:03.719365 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bf9da-98b2-4859-b456-7d4f4dbcc4ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:03.721094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bf9da-98b2-4859-b456-7d4f4dbcc4ef None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:03.740618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 173/333] 10.222.0.79 () {56 vars in 1308 bytes} [Tue Sep 6 21:00:03 2022] GET /identity/v3/role_assignments?user.id=cb533823e6e74886979475f9399129e4&scope.system=all&role.id=180edbb8a9e74555af89d6a7639f819a => generated 510 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:03.804606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed09365-1fa5-4c86-9408-d12cf738f082 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:03.825490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:03.848849 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94df49-9725-4f42-94b1-b2ad701cf204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:03.849548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94df49-9725-4f42-94b1-b2ad701cf204 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:03.849828 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94df49-9725-4f42-94b1-b2ad701cf204 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:03.850103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94df49-9725-4f42-94b1-b2ad701cf204 None admin] PATH_INFO: `/v3/system/users/c974cc998e1e4a2081569082d11ab599/roles/6abadafbec7f43d2b2cc2df439c44eff` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:03.851954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94df49-9725-4f42-94b1-b2ad701cf204 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c974cc998e1e4a2081569082d11ab599, role_id=6abadafbec7f43d2b2cc2df439c44eff)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:03.853421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94df49-9725-4f42-94b1-b2ad701cf204 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:03.864551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 161/334] 10.222.0.79 () {58 vars in 1334 bytes} [Tue Sep 6 21:00:03 2022] PUT /identity/v3/system/users/c974cc998e1e4a2081569082d11ab599/roles/6abadafbec7f43d2b2cc2df439c44eff => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:00:05.700817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06bcd7-296d-4b50-a295-bdf8190f7e9c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:05.701850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06bcd7-296d-4b50-a295-bdf8190f7e9c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:05.702327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06bcd7-296d-4b50-a295-bdf8190f7e9c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:05.704262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 174/335] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:05.710164 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55cba5-d1c8-42ce-9cb7-de3856266cae None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:05.710680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55cba5-d1c8-42ce-9cb7-de3856266cae None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:05.710680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55cba5-d1c8-42ce-9cb7-de3856266cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:05.735042 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9a55cba5-d1c8-42ce-9cb7-de3856266cae None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:05.736278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9a55cba5-d1c8-42ce-9cb7-de3856266cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:05.847832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 162/336] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:05.970861 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77f271-09ee-4b3e-add0-a21247d08e7b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:05.971207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77f271-09ee-4b3e-add0-a21247d08e7b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:05.971412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77f271-09ee-4b3e-add0-a21247d08e7b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:05.973042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 175/337] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:05.995677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f68cf-29a1-42e7-b8c0-ed37d3dbae42 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:05.995990 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f68cf-29a1-42e7-b8c0-ed37d3dbae42 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:05.996412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f68cf-29a1-42e7-b8c0-ed37d3dbae42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:06.025794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7c2f68cf-29a1-42e7-b8c0-ed37d3dbae42 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:06.027006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7c2f68cf-29a1-42e7-b8c0-ed37d3dbae42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:06.110597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 163/338] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:06.450055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77f271-09ee-4b3e-add0-a21247d08e7b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:06.506256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f19d1-f9b5-4cbd-bb30-512ef87f1fd4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:06.506598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f19d1-f9b5-4cbd-bb30-512ef87f1fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:06.506816 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f19d1-f9b5-4cbd-bb30-512ef87f1fd4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:06.508435 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 164/339] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:06.541921 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aec76-e47b-45e5-baae-5b5e3c78a4d4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:06.542266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aec76-e47b-45e5-baae-5b5e3c78a4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:06.542507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aec76-e47b-45e5-baae-5b5e3c78a4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:06.573818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2a5aec76-e47b-45e5-baae-5b5e3c78a4d4 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:06.575037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2a5aec76-e47b-45e5-baae-5b5e3c78a4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:06.606130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:06.617363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77f271-09ee-4b3e-add0-a21247d08e7b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:06.647703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:06.692792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d7077-640d-453a-8a9a-ba3f9ba154c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:06.693441 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d7077-640d-453a-8a9a-ba3f9ba154c6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:06.693715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d7077-640d-453a-8a9a-ba3f9ba154c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:06.693984 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d7077-640d-453a-8a9a-ba3f9ba154c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:06.711667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 165/340] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 21:00:06.713881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5aec76-e47b-45e5-baae-5b5e3c78a4d4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:06.729586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d7077-640d-453a-8a9a-ba3f9ba154c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c77d7077-640d-453a-8a9a-ba3f9ba154c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:06.734754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:06.735700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:06.736550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:06.736550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:06.736550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:06.736550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:06.736550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:06.736550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:06.736550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:06.736550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:06.736550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:06.736550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:06.736550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:06.736550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:06.736550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 176/341] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:00:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:06.744694 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d7077-640d-453a-8a9a-ba3f9ba154c6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:06.908919 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5aec76-e47b-45e5-baae-5b5e3c78a4d4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:06.968237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41eba9-f151-4554-a9f0-c329bc0dc707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:06.968836 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41eba9-f151-4554-a9f0-c329bc0dc707 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:06.969110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41eba9-f151-4554-a9f0-c329bc0dc707 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:06.969371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41eba9-f151-4554-a9f0-c329bc0dc707 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:06.971605 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d7077-640d-453a-8a9a-ba3f9ba154c6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:07.005340 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41eba9-f151-4554-a9f0-c329bc0dc707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-dd41eba9-f151-4554-a9f0-c329bc0dc707 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:07.010780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:07.011682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:07.012488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:07.012488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:07.012488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:07.012488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:07.012488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:07.012488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:07.012488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:07.012488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:07.012488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:07.012488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:07.012488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:07.012488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:07.012488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 166/342] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:07.013874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee9b63-743f-49ec-a786-c8747a1830ab None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:07.014065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee9b63-743f-49ec-a786-c8747a1830ab None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:07.014388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee9b63-743f-49ec-a786-c8747a1830ab None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:07.015791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 167/343] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:07.018108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee9b63-743f-49ec-a786-c8747a1830ab None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:07.019121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f02ff8-2795-47ad-92fa-af2097e643e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:07.019792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f02ff8-2795-47ad-92fa-af2097e643e9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:07.020074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f02ff8-2795-47ad-92fa-af2097e643e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:07.020341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f02ff8-2795-47ad-92fa-af2097e643e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:07.021071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f02ff8-2795-47ad-92fa-af2097e643e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:07.021371 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f02ff8-2795-47ad-92fa-af2097e643e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:07.022767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f02ff8-2795-47ad-92fa-af2097e643e9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:07.032069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 177/344] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:07.033388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1195bb7-2c96-4742-bf35-c773f40a7a76 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:07.033667 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1195bb7-2c96-4742-bf35-c773f40a7a76 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:07.033937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1195bb7-2c96-4742-bf35-c773f40a7a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:07.067614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c1195bb7-2c96-4742-bf35-c773f40a7a76 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:07.071908 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c1195bb7-2c96-4742-bf35-c773f40a7a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:07.089060 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:07.118743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:07.185128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 178/345] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:07.186406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1195bb7-2c96-4742-bf35-c773f40a7a76 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:07.195275 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee9b63-743f-49ec-a786-c8747a1830ab None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:07.245071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c220f-bfbf-4941-8aef-07113bd37a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:07.246242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c220f-bfbf-4941-8aef-07113bd37a67 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:07.247169 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c220f-bfbf-4941-8aef-07113bd37a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:07.247671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c220f-bfbf-4941-8aef-07113bd37a67 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:07.248929 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c220f-bfbf-4941-8aef-07113bd37a67 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:07.249670 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c220f-bfbf-4941-8aef-07113bd37a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:07.251699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c220f-bfbf-4941-8aef-07113bd37a67 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:07.264415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 168/346] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 21:00:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:07.265793 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c220f-bfbf-4941-8aef-07113bd37a67 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:07.319282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1195bb7-2c96-4742-bf35-c773f40a7a76 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:07.369397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab34fd-65b3-4cee-8909-0388302f3098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:07.370424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab34fd-65b3-4cee-8909-0388302f3098 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:07.370903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab34fd-65b3-4cee-8909-0388302f3098 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:07.371363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab34fd-65b3-4cee-8909-0388302f3098 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:07.421037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:07.424214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:07.440570 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c220f-bfbf-4941-8aef-07113bd37a67 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:07.492121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab34fd-65b3-4cee-8909-0388302f3098 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:07.493885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab34fd-65b3-4cee-8909-0388302f3098 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:07.510129 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0ea64-4e4e-42da-bef8-147f794f636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:07.510780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0ea64-4e4e-42da-bef8-147f794f636c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:07.511157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0ea64-4e4e-42da-bef8-147f794f636c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:07.511562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0ea64-4e4e-42da-bef8-147f794f636c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:07.534116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0ea64-4e4e-42da-bef8-147f794f636c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1cd0ea64-4e4e-42da-bef8-147f794f636c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:07.539598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:07.540498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:07.541364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:07.541364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:07.541364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:07.541364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:07.541364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:07.541364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:07.541364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:07.541364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:07.541364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:07.541364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:07.541364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 21:00:07.541364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:07.541364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 169/347] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:07.542029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0ea64-4e4e-42da-bef8-147f794f636c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:07.614351 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 179/348] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:07.615400 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab34fd-65b3-4cee-8909-0388302f3098 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:07.675839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0ea64-4e4e-42da-bef8-147f794f636c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:07.676406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:07.712969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795f010-73cb-4cf4-9a92-88970318d673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:07.713760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795f010-73cb-4cf4-9a92-88970318d673 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:07.714047 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795f010-73cb-4cf4-9a92-88970318d673 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:07.714325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795f010-73cb-4cf4-9a92-88970318d673 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:07.735109 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab34fd-65b3-4cee-8909-0388302f3098 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:07.739980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:07.780912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc5a07-5e17-4644-9c81-9e1e5450e5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:07.781455 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc5a07-5e17-4644-9c81-9e1e5450e5dd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:07.781727 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc5a07-5e17-4644-9c81-9e1e5450e5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:07.781994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc5a07-5e17-4644-9c81-9e1e5450e5dd None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:07.782879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc5a07-5e17-4644-9c81-9e1e5450e5dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:07.783258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc5a07-5e17-4644-9c81-9e1e5450e5dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:07.784853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc5a07-5e17-4644-9c81-9e1e5450e5dd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:07.793775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795f010-73cb-4cf4-9a92-88970318d673 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:07.794680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 180/349] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 21:00:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:07.795630 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795f010-73cb-4cf4-9a92-88970318d673 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:07.800294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc5a07-5e17-4644-9c81-9e1e5450e5dd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:07.907332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 170/350] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:07.908212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795f010-73cb-4cf4-9a92-88970318d673 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:07.913313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc5a07-5e17-4644-9c81-9e1e5450e5dd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:07.931048 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:07.958136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c441f14-604b-4207-9f82-cb97c7b70011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:07.959266 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c441f14-604b-4207-9f82-cb97c7b70011 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:07.959729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c441f14-604b-4207-9f82-cb97c7b70011 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:07.960172 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c441f14-604b-4207-9f82-cb97c7b70011 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:07.972587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c441f14-604b-4207-9f82-cb97c7b70011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4c441f14-604b-4207-9f82-cb97c7b70011 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:07.979657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:07.980824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:07.981833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:07.981833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:07.981833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:07.981833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:07.981833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:07.981833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:07.981833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:07.981833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:07.981833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:07.981833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:07.981833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:07.981833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:07.983229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 181/351] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:07.987984 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c441f14-604b-4207-9f82-cb97c7b70011 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:07.989077 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:08.044569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795f010-73cb-4cf4-9a92-88970318d673 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:08.093436 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40c9dc-18ec-420a-a1dd-6f876a8148ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:08.094086 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40c9dc-18ec-420a-a1dd-6f876a8148ef None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:08.094725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40c9dc-18ec-420a-a1dd-6f876a8148ef None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:08.094725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40c9dc-18ec-420a-a1dd-6f876a8148ef None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:08.095781 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40c9dc-18ec-420a-a1dd-6f876a8148ef None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:08.100569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40c9dc-18ec-420a-a1dd-6f876a8148ef None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:08.100569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 171/352] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:07 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:08.100569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d57da4-3d46-47aa-937d-9cbf9cb9520e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:08.100569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d57da4-3d46-47aa-937d-9cbf9cb9520e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:08.101035 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d57da4-3d46-47aa-937d-9cbf9cb9520e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:08.104584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 172/353] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:08.105406 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d57da4-3d46-47aa-937d-9cbf9cb9520e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:08.114803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c441f14-604b-4207-9f82-cb97c7b70011 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:08.170430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ba2da-2383-4a27-b566-202a176396cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:08.171306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ba2da-2383-4a27-b566-202a176396cc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:08.171423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ba2da-2383-4a27-b566-202a176396cc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:08.171659 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ba2da-2383-4a27-b566-202a176396cc None admin] PATH_INFO: `/v3/users/c974cc998e1e4a2081569082d11ab599` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:08.172967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ba2da-2383-4a27-b566-202a176396cc None admin] RBAC: Authorizing `identity:get_user(user_id=c974cc998e1e4a2081569082d11ab599)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:08.174291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ba2da-2383-4a27-b566-202a176396cc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:08.175867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 182/354] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:00:07 2022] GET /identity/v3/users/c974cc998e1e4a2081569082d11ab599 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:08.176773 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ba2da-2383-4a27-b566-202a176396cc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:08.237759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:08.249456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d57da4-3d46-47aa-937d-9cbf9cb9520e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:08.277103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:08.295184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70905b-1b35-40fa-bd32-710c4e1231b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:08.296321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70905b-1b35-40fa-bd32-710c4e1231b5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:08.296321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70905b-1b35-40fa-bd32-710c4e1231b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:08.296321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70905b-1b35-40fa-bd32-710c4e1231b5 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:08.297566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70905b-1b35-40fa-bd32-710c4e1231b5 None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:08.299011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70905b-1b35-40fa-bd32-710c4e1231b5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:08.303438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 173/355] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:08 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:08.303564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ba2da-2383-4a27-b566-202a176396cc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:08.303640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70905b-1b35-40fa-bd32-710c4e1231b5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:08.345631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e05d6-1be1-4299-9ab2-8d17bb94ee72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:08.346288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e05d6-1be1-4299-9ab2-8d17bb94ee72 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:08.346566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e05d6-1be1-4299-9ab2-8d17bb94ee72 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:08.346829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e05d6-1be1-4299-9ab2-8d17bb94ee72 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:08.347486 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e05d6-1be1-4299-9ab2-8d17bb94ee72 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:08.347783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e05d6-1be1-4299-9ab2-8d17bb94ee72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:08.350353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e05d6-1be1-4299-9ab2-8d17bb94ee72 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:08.359443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 183/356] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 21:00:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:08.360975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e7d0b-0108-4aa1-b3be-5f3f805ea357 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:08.361093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e7d0b-0108-4aa1-b3be-5f3f805ea357 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:08.361263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e7d0b-0108-4aa1-b3be-5f3f805ea357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:08.383171 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4c4e7d0b-0108-4aa1-b3be-5f3f805ea357 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:08.384921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4c4e7d0b-0108-4aa1-b3be-5f3f805ea357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:08.433495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70905b-1b35-40fa-bd32-710c4e1231b5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:08.466559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 184/357] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:08.467429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e7d0b-0108-4aa1-b3be-5f3f805ea357 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:08.478906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f284c-9c64-4dec-91f6-a62b95652e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:08.479547 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f284c-9c64-4dec-91f6-a62b95652e16 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:08.480136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f284c-9c64-4dec-91f6-a62b95652e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:08.480407 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f284c-9c64-4dec-91f6-a62b95652e16 None admin] PATH_INFO: `/v3/groups/09b16cf2af00486cb37e47678e78fba2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:08.488725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f284c-9c64-4dec-91f6-a62b95652e16 None admin] RBAC: Authorizing `identity:get_group(group_id=09b16cf2af00486cb37e47678e78fba2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:08.490212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f284c-9c64-4dec-91f6-a62b95652e16 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:08.492054 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 174/358] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:08 2022] GET /identity/v3/groups/09b16cf2af00486cb37e47678e78fba2 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:08.496417 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff0931-554c-41bb-b909-f3319525cfd2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:08.496417 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff0931-554c-41bb-b909-f3319525cfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:08.496417 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff0931-554c-41bb-b909-f3319525cfd2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:08.496417 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 175/359] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:08.496417 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff0931-554c-41bb-b909-f3319525cfd2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:08.527247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:08.564672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:08.618015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e7d0b-0108-4aa1-b3be-5f3f805ea357 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:08.632858 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff0931-554c-41bb-b909-f3319525cfd2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:08.672446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d507de-f7f8-46a2-a13c-bf54fc78ee61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:08.673093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d507de-f7f8-46a2-a13c-bf54fc78ee61 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:08.673591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d507de-f7f8-46a2-a13c-bf54fc78ee61 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:08.673749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d507de-f7f8-46a2-a13c-bf54fc78ee61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:08.674553 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d507de-f7f8-46a2-a13c-bf54fc78ee61 None admin] RBAC: Adding query filter params (role.id=6abadafbec7f43d2b2cc2df439c44eff, user.id=c974cc998e1e4a2081569082d11ab599, scope.system=all) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:08.674859 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d507de-f7f8-46a2-a13c-bf54fc78ee61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:08.675833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08836fb2-4818-4dcf-9e3d-e35e3ee3c793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:08.676427 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d507de-f7f8-46a2-a13c-bf54fc78ee61 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:08.676526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08836fb2-4818-4dcf-9e3d-e35e3ee3c793 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:08.676771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08836fb2-4818-4dcf-9e3d-e35e3ee3c793 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:08.677055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08836fb2-4818-4dcf-9e3d-e35e3ee3c793 None admin] PATH_INFO: `/v3/groups/09b16cf2af00486cb37e47678e78fba2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:08.678265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08836fb2-4818-4dcf-9e3d-e35e3ee3c793 None admin] RBAC: Authorizing `identity:get_group(group_id=09b16cf2af00486cb37e47678e78fba2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:08.679634 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08836fb2-4818-4dcf-9e3d-e35e3ee3c793 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:08.682313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 176/360] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:08 2022] GET /identity/v3/groups/09b16cf2af00486cb37e47678e78fba2 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:08.682937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08836fb2-4818-4dcf-9e3d-e35e3ee3c793 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:08.691574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 185/361] 10.222.0.79 () {56 vars in 1308 bytes} [Tue Sep 6 21:00:08 2022] GET /identity/v3/role_assignments?user.id=c974cc998e1e4a2081569082d11ab599&scope.system=all&role.id=6abadafbec7f43d2b2cc2df439c44eff => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:08.692593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d507de-f7f8-46a2-a13c-bf54fc78ee61 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:08.805956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08836fb2-4818-4dcf-9e3d-e35e3ee3c793 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:08.816502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:08.834696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d507de-f7f8-46a2-a13c-bf54fc78ee61 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:08.838786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:08.853614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c32243-8749-4b35-bcb1-7293893ac817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:08.854249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32243-8749-4b35-bcb1-7293893ac817 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:08.854529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32243-8749-4b35-bcb1-7293893ac817 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:08.854800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32243-8749-4b35-bcb1-7293893ac817 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:08.898270 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e94de8-9b2f-406d-992a-8851a4e2b443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:08.898956 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e94de8-9b2f-406d-992a-8851a4e2b443 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:08.899224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e94de8-9b2f-406d-992a-8851a4e2b443 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:08.899483 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e94de8-9b2f-406d-992a-8851a4e2b443 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:08.900250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e94de8-9b2f-406d-992a-8851a4e2b443 None admin] RBAC: Adding query filter params (group.id=09b16cf2af00486cb37e47678e78fba2, role.id=e945f087a5bc4c38bd8848782f76c214, scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:08.900527 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e94de8-9b2f-406d-992a-8851a4e2b443 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:08.902051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e94de8-9b2f-406d-992a-8851a4e2b443 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:08.910260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 186/362] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Sep 6 21:00:08 2022] GET /identity/v3/role_assignments?group.id=09b16cf2af00486cb37e47678e78fba2&scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e&role.id=e945f087a5bc4c38bd8848782f76c214 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:08.913217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ebf53-6534-4aa1-929a-f98d9ad912a1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:08.913497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ebf53-6534-4aa1-929a-f98d9ad912a1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:08.913768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ebf53-6534-4aa1-929a-f98d9ad912a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:08.938085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0d3ebf53-6534-4aa1-929a-f98d9ad912a1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:08.939305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0d3ebf53-6534-4aa1-929a-f98d9ad912a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:08.988916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c32243-8749-4b35-bcb1-7293893ac817 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:08.990533 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c32243-8749-4b35-bcb1-7293893ac817 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:09.032010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 187/363] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:09.037968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ebf53-6534-4aa1-929a-f98d9ad912a1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:09.128027 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 177/364] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:09.128027 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c32243-8749-4b35-bcb1-7293893ac817 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:09.161333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:09.166828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:09.190538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ebf53-6534-4aa1-929a-f98d9ad912a1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:09.230479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c77254-2dcf-47a5-b9f4-324c61160993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:09.231452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c77254-2dcf-47a5-b9f4-324c61160993 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:09.232003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c77254-2dcf-47a5-b9f4-324c61160993 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:09.232465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c77254-2dcf-47a5-b9f4-324c61160993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:09.233425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c77254-2dcf-47a5-b9f4-324c61160993 None admin] RBAC: Adding query filter params (group.id=09b16cf2af00486cb37e47678e78fba2, role.id=180edbb8a9e74555af89d6a7639f819a, scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:09.233907 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c77254-2dcf-47a5-b9f4-324c61160993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:09.235736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c77254-2dcf-47a5-b9f4-324c61160993 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:09.242528 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 188/365] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Sep 6 21:00:09 2022] GET /identity/v3/role_assignments?group.id=09b16cf2af00486cb37e47678e78fba2&scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e&role.id=180edbb8a9e74555af89d6a7639f819a => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:09.243726 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c77254-2dcf-47a5-b9f4-324c61160993 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:09.266107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c32243-8749-4b35-bcb1-7293893ac817 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:09.305632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d18903-4ebd-4882-b9d3-e6803e9bdb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:09.306680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d18903-4ebd-4882-b9d3-e6803e9bdb18 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:09.306680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d18903-4ebd-4882-b9d3-e6803e9bdb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:09.306909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d18903-4ebd-4882-b9d3-e6803e9bdb18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:09.319881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d18903-4ebd-4882-b9d3-e6803e9bdb18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-86d18903-4ebd-4882-b9d3-e6803e9bdb18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:09.328687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:09.329783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:09.330562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:09.330562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:09.330562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:09.330562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:09.330562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:09.330562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:09.330562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:09.330562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:09.330562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:09.330562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:09.330562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:09.330562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:09.330562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 178/366] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:00:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:09.332992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d18903-4ebd-4882-b9d3-e6803e9bdb18 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:09.412230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c77254-2dcf-47a5-b9f4-324c61160993 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:09.444865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:09.466406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497879b-c86e-459b-8ea5-1df69e600d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:09.467448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497879b-c86e-459b-8ea5-1df69e600d21 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:09.468150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497879b-c86e-459b-8ea5-1df69e600d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:09.468572 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497879b-c86e-459b-8ea5-1df69e600d21 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:09.470568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497879b-c86e-459b-8ea5-1df69e600d21 None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:09.471738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d18903-4ebd-4882-b9d3-e6803e9bdb18 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:09.472358 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497879b-c86e-459b-8ea5-1df69e600d21 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:09.474872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 189/367] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:09 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:09.476824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497879b-c86e-459b-8ea5-1df69e600d21 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:09.505789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:09.523921 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed5ff6-8aa8-45c3-8c53-08fd735dd50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:09.525158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed5ff6-8aa8-45c3-8c53-08fd735dd50f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:09.525715 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed5ff6-8aa8-45c3-8c53-08fd735dd50f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:09.527143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed5ff6-8aa8-45c3-8c53-08fd735dd50f None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:09.528017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed5ff6-8aa8-45c3-8c53-08fd735dd50f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:09.528503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed5ff6-8aa8-45c3-8c53-08fd735dd50f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:09.530198 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed5ff6-8aa8-45c3-8c53-08fd735dd50f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:09.542573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 179/368] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:09.544111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed5ff6-8aa8-45c3-8c53-08fd735dd50f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:09.603298 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497879b-c86e-459b-8ea5-1df69e600d21 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:09.643119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c16ee-4e29-4a74-9955-437466434157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:09.643791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c16ee-4e29-4a74-9955-437466434157 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:09.644110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c16ee-4e29-4a74-9955-437466434157 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:09.644381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c16ee-4e29-4a74-9955-437466434157 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:09.656813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c16ee-4e29-4a74-9955-437466434157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6a6c16ee-4e29-4a74-9955-437466434157 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:09.662463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:09.663339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:09.664103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:09.664103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:09.664103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:09.664103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:09.664103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:09.664103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:09.664103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:09.664103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:09.664103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:09.664103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:09.664103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:00:09.664103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:09.664103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 190/369] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:00:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:09.664800 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c16ee-4e29-4a74-9955-437466434157 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:09.679244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed5ff6-8aa8-45c3-8c53-08fd735dd50f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:09.719625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ea7e0-1fd6-4d5a-81d5-ee7c9376e82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:09.720270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ea7e0-1fd6-4d5a-81d5-ee7c9376e82d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:09.720548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ea7e0-1fd6-4d5a-81d5-ee7c9376e82d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:09.720852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ea7e0-1fd6-4d5a-81d5-ee7c9376e82d None admin] PATH_INFO: `/v3/groups/09b16cf2af00486cb37e47678e78fba2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:09.722055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ea7e0-1fd6-4d5a-81d5-ee7c9376e82d None admin] RBAC: Authorizing `identity:get_group(group_id=09b16cf2af00486cb37e47678e78fba2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:09.723566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ea7e0-1fd6-4d5a-81d5-ee7c9376e82d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:09.726263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 180/370] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:09 2022] GET /identity/v3/groups/09b16cf2af00486cb37e47678e78fba2 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:09.727107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ea7e0-1fd6-4d5a-81d5-ee7c9376e82d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:09.740557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:09.773277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:09.804584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c16ee-4e29-4a74-9955-437466434157 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:09.852363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ea7e0-1fd6-4d5a-81d5-ee7c9376e82d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:09.862947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db8c00-7218-454e-b5dc-38b46e85a269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:09.863617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db8c00-7218-454e-b5dc-38b46e85a269 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:09.863900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db8c00-7218-454e-b5dc-38b46e85a269 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:09.864180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db8c00-7218-454e-b5dc-38b46e85a269 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:09.898626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ef970-ddaa-4b11-9a77-8fae242224b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:09.899295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ef970-ddaa-4b11-9a77-8fae242224b9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:09.899576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ef970-ddaa-4b11-9a77-8fae242224b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:09.900020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ef970-ddaa-4b11-9a77-8fae242224b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:09.900784 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ef970-ddaa-4b11-9a77-8fae242224b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:09.901421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ef970-ddaa-4b11-9a77-8fae242224b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:09.903178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ef970-ddaa-4b11-9a77-8fae242224b9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:09.917178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 181/371] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:09.921496 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ef970-ddaa-4b11-9a77-8fae242224b9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:09.961558 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db8c00-7218-454e-b5dc-38b46e85a269 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:09.963248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db8c00-7218-454e-b5dc-38b46e85a269 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:10.002502 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:10.036171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ef970-ddaa-4b11-9a77-8fae242224b9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:10.054277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:10.071789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c41894-a49d-4ceb-b797-adfd4aff0750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:10.072409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c41894-a49d-4ceb-b797-adfd4aff0750 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:10.072691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c41894-a49d-4ceb-b797-adfd4aff0750 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:10.072971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c41894-a49d-4ceb-b797-adfd4aff0750 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:10.073764 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c41894-a49d-4ceb-b797-adfd4aff0750 None admin] RBAC: Adding query filter params (group.id=09b16cf2af00486cb37e47678e78fba2, role.id=180edbb8a9e74555af89d6a7639f819a, scope.project.id=402258177cb849268ec3155d32bedd84) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:10.074075 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c41894-a49d-4ceb-b797-adfd4aff0750 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:10.075599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c41894-a49d-4ceb-b797-adfd4aff0750 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:10.081795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 182/372] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Sep 6 21:00:09 2022] GET /identity/v3/role_assignments?group.id=09b16cf2af00486cb37e47678e78fba2&scope.project.id=402258177cb849268ec3155d32bedd84&role.id=180edbb8a9e74555af89d6a7639f819a => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:10.082572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c41894-a49d-4ceb-b797-adfd4aff0750 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:10.085659 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 191/373] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:10.093406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db8c00-7218-454e-b5dc-38b46e85a269 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:10.229456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db8c00-7218-454e-b5dc-38b46e85a269 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:10.236954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c41894-a49d-4ceb-b797-adfd4aff0750 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:10.304874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae172ce-ae90-46b4-9a49-c70aa8b91b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:10.305933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0816092-92bb-4340-9ad3-810dd743ae27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:10.306794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0816092-92bb-4340-9ad3-810dd743ae27 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:10.307712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae172ce-ae90-46b4-9a49-c70aa8b91b8e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:10.308204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0816092-92bb-4340-9ad3-810dd743ae27 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:10.308708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0816092-92bb-4340-9ad3-810dd743ae27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:10.309187 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae172ce-ae90-46b4-9a49-c70aa8b91b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:10.310349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae172ce-ae90-46b4-9a49-c70aa8b91b8e None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:10.314377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae172ce-ae90-46b4-9a49-c70aa8b91b8e None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:10.316513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae172ce-ae90-46b4-9a49-c70aa8b91b8e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:10.319360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 192/374] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:10 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:10.319987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:10.326815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae172ce-ae90-46b4-9a49-c70aa8b91b8e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:10.429778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0816092-92bb-4340-9ad3-810dd743ae27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:10.431394 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0816092-92bb-4340-9ad3-810dd743ae27 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:10.433003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:10.502281 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae172ce-ae90-46b4-9a49-c70aa8b91b8e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:10.541521 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f569f5-2490-4be8-b10d-e35826a86b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:10.542193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f569f5-2490-4be8-b10d-e35826a86b00 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:10.542471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f569f5-2490-4be8-b10d-e35826a86b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:10.542741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f569f5-2490-4be8-b10d-e35826a86b00 None admin] PATH_INFO: `/v3/groups/09b16cf2af00486cb37e47678e78fba2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:10.543943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f569f5-2490-4be8-b10d-e35826a86b00 None admin] RBAC: Authorizing `identity:get_group(group_id=09b16cf2af00486cb37e47678e78fba2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:10.545410 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f569f5-2490-4be8-b10d-e35826a86b00 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:10.547303 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 193/375] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:10 2022] GET /identity/v3/groups/09b16cf2af00486cb37e47678e78fba2 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:10.552806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f569f5-2490-4be8-b10d-e35826a86b00 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:10.654214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 183/376] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:10.659935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0816092-92bb-4340-9ad3-810dd743ae27 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:10.668672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f569f5-2490-4be8-b10d-e35826a86b00 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:10.686721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:10.702626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:10.721014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd53f7-f508-435f-834b-a4330c8a4eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:10.721630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd53f7-f508-435f-834b-a4330c8a4eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:10.721903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd53f7-f508-435f-834b-a4330c8a4eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:10.722166 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd53f7-f508-435f-834b-a4330c8a4eb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:10.722891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd53f7-f508-435f-834b-a4330c8a4eb1 None admin] RBAC: Adding query filter params (group.id=09b16cf2af00486cb37e47678e78fba2, role.id=e945f087a5bc4c38bd8848782f76c214, scope.project.id=402258177cb849268ec3155d32bedd84) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:10.723193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd53f7-f508-435f-834b-a4330c8a4eb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:10.725872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd53f7-f508-435f-834b-a4330c8a4eb1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:10.740167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 194/377] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Sep 6 21:00:10 2022] GET /identity/v3/role_assignments?group.id=09b16cf2af00486cb37e47678e78fba2&scope.project.id=402258177cb849268ec3155d32bedd84&role.id=e945f087a5bc4c38bd8848782f76c214 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:10.794435 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0816092-92bb-4340-9ad3-810dd743ae27 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:10.836977 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e1443-bb1d-45a6-9e07-b11537f7855a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:10.837642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e1443-bb1d-45a6-9e07-b11537f7855a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:10.837919 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e1443-bb1d-45a6-9e07-b11537f7855a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:10.838193 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e1443-bb1d-45a6-9e07-b11537f7855a None admin] PATH_INFO: `/v3/projects/0b4b2ee27141432e8ca02dac223fb495` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:10.840966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e1443-bb1d-45a6-9e07-b11537f7855a None admin] RBAC: Authorizing `identity:get_project(project_id=0b4b2ee27141432e8ca02dac223fb495)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:10.842524 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e1443-bb1d-45a6-9e07-b11537f7855a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:10.844300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 184/378] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:10 2022] GET /identity/v3/projects/0b4b2ee27141432e8ca02dac223fb495 => generated 344 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:10.850596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd53f7-f508-435f-834b-a4330c8a4eb1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:10.981172 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd53f7-f508-435f-834b-a4330c8a4eb1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:11.023371 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7acd81-6c94-4ea0-adaf-d2c215691794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:11.024011 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7acd81-6c94-4ea0-adaf-d2c215691794 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:11.024288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7acd81-6c94-4ea0-adaf-d2c215691794 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:11.024559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7acd81-6c94-4ea0-adaf-d2c215691794 None admin] PATH_INFO: `/v3/groups/e34a622ca2ef442b8d0162423cba4f2f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:11.032338 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7acd81-6c94-4ea0-adaf-d2c215691794 None admin] RBAC: Authorizing `identity:get_group(group_id=e34a622ca2ef442b8d0162423cba4f2f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:11.033824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7acd81-6c94-4ea0-adaf-d2c215691794 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:11.036782 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 195/379] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:10 2022] GET /identity/v3/groups/e34a622ca2ef442b8d0162423cba4f2f => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:11.041794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e1443-bb1d-45a6-9e07-b11537f7855a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:11.154118 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e1443-bb1d-45a6-9e07-b11537f7855a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:11.156643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:11.197566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777acc8-056e-4aae-a6c7-6ba625a5ea34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:11.198248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777acc8-056e-4aae-a6c7-6ba625a5ea34 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:11.198525 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777acc8-056e-4aae-a6c7-6ba625a5ea34 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:11.198794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777acc8-056e-4aae-a6c7-6ba625a5ea34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:11.199594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777acc8-056e-4aae-a6c7-6ba625a5ea34 None admin] RBAC: Adding query filter params (group.id=e34a622ca2ef442b8d0162423cba4f2f, role.id=3b30504991194002b41f8587d0851c10, scope.project.id=0b4b2ee27141432e8ca02dac223fb495) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:11.199902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777acc8-056e-4aae-a6c7-6ba625a5ea34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:11.201444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777acc8-056e-4aae-a6c7-6ba625a5ea34 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:11.213129 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 185/380] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Sep 6 21:00:11 2022] GET /identity/v3/role_assignments?group.id=e34a622ca2ef442b8d0162423cba4f2f&scope.project.id=0b4b2ee27141432e8ca02dac223fb495&role.id=3b30504991194002b41f8587d0851c10 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:11.457855 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a3e06-28f0-4fcb-ad77-406be873765f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:11.458040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a3e06-28f0-4fcb-ad77-406be873765f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:11.458377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a3e06-28f0-4fcb-ad77-406be873765f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:11.460108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 196/381] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:11.465449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec33be1-868f-4801-b4f5-4311e9dc6fa1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:11.465918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec33be1-868f-4801-b4f5-4311e9dc6fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:11.466094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec33be1-868f-4801-b4f5-4311e9dc6fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:11.490740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cec33be1-868f-4801-b4f5-4311e9dc6fa1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:11.491964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cec33be1-868f-4801-b4f5-4311e9dc6fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:11.572021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 186/382] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:12.084121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a3e06-28f0-4fcb-ad77-406be873765f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:12.134596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:12.178830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd83481-c993-4358-a6e6-7f9011904f84 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:12.179361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd83481-c993-4358-a6e6-7f9011904f84 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:12.179361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd83481-c993-4358-a6e6-7f9011904f84 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:12.180985 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 187/383] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:12.187444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481d1a-edf5-428d-9d92-ab1db4473a90 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:12.187730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481d1a-edf5-428d-9d92-ab1db4473a90 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:12.188005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481d1a-edf5-428d-9d92-ab1db4473a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:12.204236 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a3e06-28f0-4fcb-ad77-406be873765f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:12.226240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-19481d1a-edf5-428d-9d92-ab1db4473a90 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:12.228367 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-19481d1a-edf5-428d-9d92-ab1db4473a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:12.276542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947de0f4-763f-4f24-8e14-5c4e45643e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:12.277545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947de0f4-763f-4f24-8e14-5c4e45643e37 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:12.278498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947de0f4-763f-4f24-8e14-5c4e45643e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:12.280877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947de0f4-763f-4f24-8e14-5c4e45643e37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:12.301117 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947de0f4-763f-4f24-8e14-5c4e45643e37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-947de0f4-763f-4f24-8e14-5c4e45643e37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:12.306785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:12.307839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:12.308817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:12.308817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:12.308817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:12.308817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:12.308817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:12.308817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:12.308817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:12.308817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:12.308817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:12.308817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:12.308817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:12.308817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:12.320234 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 197/384] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:00:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:12.326560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947de0f4-763f-4f24-8e14-5c4e45643e37 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:12.356825 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 188/385] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 21:00:12.448609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947de0f4-763f-4f24-8e14-5c4e45643e37 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:12.450290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:12.499248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15ff47-227d-4bdc-a2b6-d4707df759f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:12.499865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15ff47-227d-4bdc-a2b6-d4707df759f4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:12.500138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15ff47-227d-4bdc-a2b6-d4707df759f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:12.500406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15ff47-227d-4bdc-a2b6-d4707df759f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:12.501097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15ff47-227d-4bdc-a2b6-d4707df759f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:12.501394 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15ff47-227d-4bdc-a2b6-d4707df759f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:12.502781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15ff47-227d-4bdc-a2b6-d4707df759f4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:12.511776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 198/386] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:12.517465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19481d1a-edf5-428d-9d92-ab1db4473a90 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:12.552862 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:12.637591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19481d1a-edf5-428d-9d92-ab1db4473a90 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:12.687265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fec463-341b-4a3a-a133-da406a9f719b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:12.687917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fec463-341b-4a3a-a133-da406a9f719b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:12.688190 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fec463-341b-4a3a-a133-da406a9f719b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:12.688492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fec463-341b-4a3a-a133-da406a9f719b None admin] PATH_INFO: `/v3/groups/09b16cf2af00486cb37e47678e78fba2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:12.689696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fec463-341b-4a3a-a133-da406a9f719b None admin] RBAC: Authorizing `identity:get_group(group_id=09b16cf2af00486cb37e47678e78fba2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:12.691155 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fec463-341b-4a3a-a133-da406a9f719b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:12.692824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 189/387] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:12 2022] GET /identity/v3/groups/09b16cf2af00486cb37e47678e78fba2 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:12.698356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15ff47-227d-4bdc-a2b6-d4707df759f4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:12.797829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fec463-341b-4a3a-a133-da406a9f719b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:12.836340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15ff47-227d-4bdc-a2b6-d4707df759f4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:12.905594 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2025c-6a93-42be-abde-aa1820c1da2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:12.906150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2025c-6a93-42be-abde-aa1820c1da2d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:12.906424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2025c-6a93-42be-abde-aa1820c1da2d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:12.906689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2025c-6a93-42be-abde-aa1820c1da2d None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:12.909880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2025c-6a93-42be-abde-aa1820c1da2d None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:12.911358 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2025c-6a93-42be-abde-aa1820c1da2d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:12.913111 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 199/388] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:12 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:12.924324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2025c-6a93-42be-abde-aa1820c1da2d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:12.983125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:12.989316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:13.001743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fec463-341b-4a3a-a133-da406a9f719b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:13.042340 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ccadc-f173-4ee6-a891-42cbc3429c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:13.043295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ccadc-f173-4ee6-a891-42cbc3429c76 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:13.043880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ccadc-f173-4ee6-a891-42cbc3429c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:13.044252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ccadc-f173-4ee6-a891-42cbc3429c76 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:13.058338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ccadc-f173-4ee6-a891-42cbc3429c76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-405ccadc-f173-4ee6-a891-42cbc3429c76 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:13.064452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:13.065484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:13.066341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:13.066341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:13.066341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:13.066341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:13.066341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:13.066341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:13.066341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:13.066341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:13.066341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:13.066341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:13.066341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:13.066341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:13.070523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 190/389] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:13.071961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dfadb-3fc7-40ca-bb6a-6d101bfaa3a7 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:13.072365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dfadb-3fc7-40ca-bb6a-6d101bfaa3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:13.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dfadb-3fc7-40ca-bb6a-6d101bfaa3a7 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:13.073319 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2025c-6a93-42be-abde-aa1820c1da2d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:13.074325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 191/390] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:13.079405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33dfadb-3fc7-40ca-bb6a-6d101bfaa3a7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:13.123274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee7981-d44c-4830-b246-8673d9b6ff04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:13.123990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee7981-d44c-4830-b246-8673d9b6ff04 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:13.124169 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee7981-d44c-4830-b246-8673d9b6ff04 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:13.124483 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee7981-d44c-4830-b246-8673d9b6ff04 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e/groups/09b16cf2af00486cb37e47678e78fba2/roles/e945f087a5bc4c38bd8848782f76c214` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:13.127315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee7981-d44c-4830-b246-8673d9b6ff04 None admin] RBAC: Authorizing `identity:create_grant(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e, group_id=09b16cf2af00486cb37e47678e78fba2, role_id=e945f087a5bc4c38bd8848782f76c214)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:13.128900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee7981-d44c-4830-b246-8673d9b6ff04 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:13.153494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 200/391] 10.222.0.79 () {58 vars in 1442 bytes} [Tue Sep 6 21:00:12 2022] PUT /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e/groups/09b16cf2af00486cb37e47678e78fba2/roles/e945f087a5bc4c38bd8848782f76c214 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:00:13.153494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23021b-d072-4392-96bf-a779a7ec7b17 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:13.153494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23021b-d072-4392-96bf-a779a7ec7b17 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:13.153494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23021b-d072-4392-96bf-a779a7ec7b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:13.173861 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9b23021b-d072-4392-96bf-a779a7ec7b17 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:13.175238 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9b23021b-d072-4392-96bf-a779a7ec7b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:13.247082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33dfadb-3fc7-40ca-bb6a-6d101bfaa3a7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:13.253501 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 201/392] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:13.306662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaba6a-aa3d-4f63-9f31-8f8de1832d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:13.307629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaba6a-aa3d-4f63-9f31-8f8de1832d01 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:13.308053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaba6a-aa3d-4f63-9f31-8f8de1832d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:13.308497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaba6a-aa3d-4f63-9f31-8f8de1832d01 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:13.309358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcaba6a-aa3d-4f63-9f31-8f8de1832d01 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:13.309848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcaba6a-aa3d-4f63-9f31-8f8de1832d01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:13.311763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcaba6a-aa3d-4f63-9f31-8f8de1832d01 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:13.329147 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 192/393] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 21:00:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:13.335277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23021b-d072-4392-96bf-a779a7ec7b17 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:13.377803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:13.440376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ceaa4f-4525-4c0e-947e-f7ad43d292e4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:13.441042 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ceaa4f-4525-4c0e-947e-f7ad43d292e4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:13.441543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ceaa4f-4525-4c0e-947e-f7ad43d292e4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:13.443508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 193/394] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:13.448932 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116294b6-eb6d-494a-b937-e8a2ce0f745f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:13.449531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116294b6-eb6d-494a-b937-e8a2ce0f745f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:13.450016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116294b6-eb6d-494a-b937-e8a2ce0f745f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:13.476239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-116294b6-eb6d-494a-b937-e8a2ce0f745f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:13.477970 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-116294b6-eb6d-494a-b937-e8a2ce0f745f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:13.485163 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23021b-d072-4392-96bf-a779a7ec7b17 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:13.495775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:13.529234 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b344a80-811e-48ae-aec7-dc2ecb68380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:13.529814 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b344a80-811e-48ae-aec7-dc2ecb68380c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:13.530091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b344a80-811e-48ae-aec7-dc2ecb68380c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:13.530361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b344a80-811e-48ae-aec7-dc2ecb68380c None admin] PATH_INFO: `/v3/groups/09b16cf2af00486cb37e47678e78fba2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:13.531931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b344a80-811e-48ae-aec7-dc2ecb68380c None admin] RBAC: Authorizing `identity:get_group(group_id=09b16cf2af00486cb37e47678e78fba2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:13.533753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b344a80-811e-48ae-aec7-dc2ecb68380c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:13.540703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 202/395] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:13 2022] GET /identity/v3/groups/09b16cf2af00486cb37e47678e78fba2 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:13.542275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b344a80-811e-48ae-aec7-dc2ecb68380c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:13.561562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 194/396] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 21:00:13.649039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:13.661391 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b344a80-811e-48ae-aec7-dc2ecb68380c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:13.682157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116294b6-eb6d-494a-b937-e8a2ce0f745f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:13.704379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee905d25-75e3-4025-b11e-b77fc08125df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:13.704960 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee905d25-75e3-4025-b11e-b77fc08125df None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:13.705237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee905d25-75e3-4025-b11e-b77fc08125df None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:13.705507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee905d25-75e3-4025-b11e-b77fc08125df None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:13.706799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee905d25-75e3-4025-b11e-b77fc08125df None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:13.708275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee905d25-75e3-4025-b11e-b77fc08125df None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:13.717749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 203/397] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:13 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:13.718602 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee905d25-75e3-4025-b11e-b77fc08125df None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:13.846226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116294b6-eb6d-494a-b937-e8a2ce0f745f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:13.896957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1835fd6-ce67-483c-b979-f0a0e6c4863d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:13.898032 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1835fd6-ce67-483c-b979-f0a0e6c4863d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:13.898510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1835fd6-ce67-483c-b979-f0a0e6c4863d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:13.899083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1835fd6-ce67-483c-b979-f0a0e6c4863d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:13.909642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee905d25-75e3-4025-b11e-b77fc08125df None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:13.913119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1835fd6-ce67-483c-b979-f0a0e6c4863d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f1835fd6-ce67-483c-b979-f0a0e6c4863d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:13.919742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:13.920720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:13.921523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:13.921523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:13.921523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:13.921523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:13.921523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:13.921523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:13.921523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:13.921523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:13.921523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:13.921523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:13.921523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:13.921523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:13.924463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 195/398] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:13.925708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1835fd6-ce67-483c-b979-f0a0e6c4863d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:13.940905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:13.960941 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d8b48-b142-4998-b379-d26f55588153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:13.962737 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d8b48-b142-4998-b379-d26f55588153 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:13.963259 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d8b48-b142-4998-b379-d26f55588153 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:13.963885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d8b48-b142-4998-b379-d26f55588153 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e/groups/09b16cf2af00486cb37e47678e78fba2/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:13.966877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d8b48-b142-4998-b379-d26f55588153 None admin] RBAC: Authorizing `identity:create_grant(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e, group_id=09b16cf2af00486cb37e47678e78fba2, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:13.969220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d8b48-b142-4998-b379-d26f55588153 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:14.000689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 204/399] 10.222.0.79 () {58 vars in 1442 bytes} [Tue Sep 6 21:00:13 2022] PUT /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e/groups/09b16cf2af00486cb37e47678e78fba2/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:00:14.000689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d8b48-b142-4998-b379-d26f55588153 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:14.047120 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:14.097439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1835fd6-ce67-483c-b979-f0a0e6c4863d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:14.151988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e98a8-e070-478a-93ab-f5cb26783088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:14.153304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e98a8-e070-478a-93ab-f5cb26783088 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:14.154117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e98a8-e070-478a-93ab-f5cb26783088 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:14.154548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e98a8-e070-478a-93ab-f5cb26783088 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:14.154997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d8b48-b142-4998-b379-d26f55588153 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:14.170523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e98a8-e070-478a-93ab-f5cb26783088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b19e98a8-e070-478a-93ab-f5cb26783088 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:14.178189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:14.179126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:14.179916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:14.179916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:14.179916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:14.179916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:14.179916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:14.179916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:14.179916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:14.179916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:14.179916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:14.179916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:14.179916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:14.179916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:14.179916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 196/400] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:00:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:14.185467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b824a-4347-4677-abbc-d1881e85aa36 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:14.185467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b824a-4347-4677-abbc-d1881e85aa36 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:14.185467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b824a-4347-4677-abbc-d1881e85aa36 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:14.185467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 197/401] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:14.190625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b824a-4347-4677-abbc-d1881e85aa36 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:14.199562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e83bf-9fe1-48ea-8a6a-164ac89d26d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:14.200226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e83bf-9fe1-48ea-8a6a-164ac89d26d8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:14.200507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e83bf-9fe1-48ea-8a6a-164ac89d26d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:14.200797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e83bf-9fe1-48ea-8a6a-164ac89d26d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:14.201536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e83bf-9fe1-48ea-8a6a-164ac89d26d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:14.201842 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e83bf-9fe1-48ea-8a6a-164ac89d26d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:14.203229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e83bf-9fe1-48ea-8a6a-164ac89d26d8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:14.215995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 205/402] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 21:00:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:14.217100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e6a31-4693-4b57-9d03-086ba263e32b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:14.217390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e6a31-4693-4b57-9d03-086ba263e32b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:14.217664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e6a31-4693-4b57-9d03-086ba263e32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:14.241307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bb3e6a31-4693-4b57-9d03-086ba263e32b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:14.242550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bb3e6a31-4693-4b57-9d03-086ba263e32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:14.273569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:14.317442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b824a-4347-4677-abbc-d1881e85aa36 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:14.334904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 206/403] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:14.336002 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e6a31-4693-4b57-9d03-086ba263e32b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:14.351676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:14.356788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0c9a2-f096-4773-9e4d-c5e384473d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:14.357850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0c9a2-f096-4773-9e4d-c5e384473d3d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:14.358377 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0c9a2-f096-4773-9e4d-c5e384473d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:14.358867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0c9a2-f096-4773-9e4d-c5e384473d3d None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:14.359845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0c9a2-f096-4773-9e4d-c5e384473d3d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:14.360443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0c9a2-f096-4773-9e4d-c5e384473d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:14.362178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0c9a2-f096-4773-9e4d-c5e384473d3d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:14.376730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 198/404] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:14.385779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0c9a2-f096-4773-9e4d-c5e384473d3d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:14.459316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e6a31-4693-4b57-9d03-086ba263e32b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:14.496276 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b02c7d-1145-4b91-8ca8-c6052b24fc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:14.497395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b02c7d-1145-4b91-8ca8-c6052b24fc36 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:14.497898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b02c7d-1145-4b91-8ca8-c6052b24fc36 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:14.498416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b02c7d-1145-4b91-8ca8-c6052b24fc36 None admin] PATH_INFO: `/v3/groups/09b16cf2af00486cb37e47678e78fba2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:14.499806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b02c7d-1145-4b91-8ca8-c6052b24fc36 None admin] RBAC: Authorizing `identity:get_group(group_id=09b16cf2af00486cb37e47678e78fba2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:14.501535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b02c7d-1145-4b91-8ca8-c6052b24fc36 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:14.503391 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 207/405] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:14 2022] GET /identity/v3/groups/09b16cf2af00486cb37e47678e78fba2 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:14.509237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b02c7d-1145-4b91-8ca8-c6052b24fc36 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:14.561117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0c9a2-f096-4773-9e4d-c5e384473d3d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:14.617473 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:14.627852 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b02c7d-1145-4b91-8ca8-c6052b24fc36 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:14.631890 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:14.642127 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc589fa-d1c4-45ca-870d-921f4e65b1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:14.643007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc589fa-d1c4-45ca-870d-921f4e65b1dd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:14.643115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc589fa-d1c4-45ca-870d-921f4e65b1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:14.643299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc589fa-d1c4-45ca-870d-921f4e65b1dd None admin] PATH_INFO: `/v3/groups/09b16cf2af00486cb37e47678e78fba2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:14.644423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc589fa-d1c4-45ca-870d-921f4e65b1dd None admin] RBAC: Authorizing `identity:get_group(group_id=09b16cf2af00486cb37e47678e78fba2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:14.645895 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc589fa-d1c4-45ca-870d-921f4e65b1dd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:14.657278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 199/406] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:14 2022] GET /identity/v3/groups/09b16cf2af00486cb37e47678e78fba2 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:14.663740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc589fa-d1c4-45ca-870d-921f4e65b1dd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:14.675062 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f36910-0f97-4b21-8e56-fa2e8e60880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:14.676084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f36910-0f97-4b21-8e56-fa2e8e60880e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:14.676623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f36910-0f97-4b21-8e56-fa2e8e60880e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:14.677187 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f36910-0f97-4b21-8e56-fa2e8e60880e None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:14.678913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f36910-0f97-4b21-8e56-fa2e8e60880e None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:14.680938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f36910-0f97-4b21-8e56-fa2e8e60880e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:14.683169 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 208/407] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:14 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:14.689832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f36910-0f97-4b21-8e56-fa2e8e60880e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:14.798560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc589fa-d1c4-45ca-870d-921f4e65b1dd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:14.799801 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f36910-0f97-4b21-8e56-fa2e8e60880e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:14.834856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c06f8a-679d-485e-9d24-e0aef66e279f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:14.835601 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c06f8a-679d-485e-9d24-e0aef66e279f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:14.835882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c06f8a-679d-485e-9d24-e0aef66e279f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:14.836154 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c06f8a-679d-485e-9d24-e0aef66e279f None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:14.837548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c06f8a-679d-485e-9d24-e0aef66e279f None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:14.839313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c06f8a-679d-485e-9d24-e0aef66e279f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:14.839606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6ee8b-e166-4f81-9a68-3ca137067167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:14.840570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6ee8b-e166-4f81-9a68-3ca137067167 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:14.841093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6ee8b-e166-4f81-9a68-3ca137067167 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:14.844685 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 200/408] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:14 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:14.844685 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c06f8a-679d-485e-9d24-e0aef66e279f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:14.845304 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6ee8b-e166-4f81-9a68-3ca137067167 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84/groups/09b16cf2af00486cb37e47678e78fba2/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:14.849341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6ee8b-e166-4f81-9a68-3ca137067167 None admin] RBAC: Authorizing `identity:create_grant(project_id=402258177cb849268ec3155d32bedd84, group_id=09b16cf2af00486cb37e47678e78fba2, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:14.851193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6ee8b-e166-4f81-9a68-3ca137067167 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:14.869933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 209/409] 10.222.0.79 () {58 vars in 1442 bytes} [Tue Sep 6 21:00:14 2022] PUT /identity/v3/projects/402258177cb849268ec3155d32bedd84/groups/09b16cf2af00486cb37e47678e78fba2/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:00:14.875000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6ee8b-e166-4f81-9a68-3ca137067167 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:14.895944 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:14.966915 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:15.035796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c06f8a-679d-485e-9d24-e0aef66e279f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:15.090237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6ee8b-e166-4f81-9a68-3ca137067167 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:15.106136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaeea7c-f1d8-4d78-b6a8-cc84775fbf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:15.106839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaeea7c-f1d8-4d78-b6a8-cc84775fbf51 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:15.107115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaeea7c-f1d8-4d78-b6a8-cc84775fbf51 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:15.107383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaeea7c-f1d8-4d78-b6a8-cc84775fbf51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:15.133547 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaeea7c-f1d8-4d78-b6a8-cc84775fbf51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-daaeea7c-f1d8-4d78-b6a8-cc84775fbf51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:15.139532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:15.140483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:15.141306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:15.141306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:15.141306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:15.141306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:15.141306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:15.141306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:15.141306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:15.141306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:15.141306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:15.141306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:15.141306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:00:15.141306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:15.144145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 201/410] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:00:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:15.151513 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaeea7c-f1d8-4d78-b6a8-cc84775fbf51 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:15.167708 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc7184-2d49-4ecd-8789-8e6788d110df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:15.169151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc7184-2d49-4ecd-8789-8e6788d110df None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:15.169629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc7184-2d49-4ecd-8789-8e6788d110df None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:15.170146 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc7184-2d49-4ecd-8789-8e6788d110df None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84/groups/09b16cf2af00486cb37e47678e78fba2/roles/e945f087a5bc4c38bd8848782f76c214` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:15.181649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc7184-2d49-4ecd-8789-8e6788d110df None admin] RBAC: Authorizing `identity:create_grant(project_id=402258177cb849268ec3155d32bedd84, group_id=09b16cf2af00486cb37e47678e78fba2, role_id=e945f087a5bc4c38bd8848782f76c214)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:15.183594 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc7184-2d49-4ecd-8789-8e6788d110df None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:15.211200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 210/411] 10.222.0.79 () {58 vars in 1442 bytes} [Tue Sep 6 21:00:14 2022] PUT /identity/v3/projects/402258177cb849268ec3155d32bedd84/groups/09b16cf2af00486cb37e47678e78fba2/roles/e945f087a5bc4c38bd8848782f76c214 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:00:15.280156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaeea7c-f1d8-4d78-b6a8-cc84775fbf51 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:15.324203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:15.336847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83422a14-0dbc-4216-a434-621056e304ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:15.337500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83422a14-0dbc-4216-a434-621056e304ab None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:15.337789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83422a14-0dbc-4216-a434-621056e304ab None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:15.338068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83422a14-0dbc-4216-a434-621056e304ab None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:15.338762 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83422a14-0dbc-4216-a434-621056e304ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:15.339512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83422a14-0dbc-4216-a434-621056e304ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:15.341090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83422a14-0dbc-4216-a434-621056e304ab None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:15.359414 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 202/412] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:15.364944 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc7184-2d49-4ecd-8789-8e6788d110df None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:15.475144 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:15.517103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc7184-2d49-4ecd-8789-8e6788d110df None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:15.560385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e375be3-686d-44ce-b940-03673f555f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:15.561502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e375be3-686d-44ce-b940-03673f555f41 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:15.561784 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e375be3-686d-44ce-b940-03673f555f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:15.562058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e375be3-686d-44ce-b940-03673f555f41 None admin] PATH_INFO: `/v3/groups/e34a622ca2ef442b8d0162423cba4f2f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:15.563334 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e375be3-686d-44ce-b940-03673f555f41 None admin] RBAC: Authorizing `identity:get_group(group_id=e34a622ca2ef442b8d0162423cba4f2f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:15.564996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e375be3-686d-44ce-b940-03673f555f41 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:15.567097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 211/413] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:15 2022] GET /identity/v3/groups/e34a622ca2ef442b8d0162423cba4f2f => generated 229 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:15.572882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83422a14-0dbc-4216-a434-621056e304ab None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:15.671386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6637ad1-bccb-477d-941f-e83483c38918 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:15.671702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6637ad1-bccb-477d-941f-e83483c38918 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:15.671942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6637ad1-bccb-477d-941f-e83483c38918 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:15.673474 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 212/414] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:00:15.679664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc5485-21a1-485a-a491-e9a4aa1492fa None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:15.680310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc5485-21a1-485a-a491-e9a4aa1492fa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:15.680823 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc5485-21a1-485a-a491-e9a4aa1492fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:15.705321 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8cbc5485-21a1-485a-a491-e9a4aa1492fa None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:15.707731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8cbc5485-21a1-485a-a491-e9a4aa1492fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:15.741677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83422a14-0dbc-4216-a434-621056e304ab None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:15.797738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6283ecc-41d5-406c-8974-e520dfc17795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:15.798771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6283ecc-41d5-406c-8974-e520dfc17795 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:15.799286 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6283ecc-41d5-406c-8974-e520dfc17795 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:15.799743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6283ecc-41d5-406c-8974-e520dfc17795 None admin] PATH_INFO: `/v3/projects/0b4b2ee27141432e8ca02dac223fb495` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:15.801217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6283ecc-41d5-406c-8974-e520dfc17795 None admin] RBAC: Authorizing `identity:get_project(project_id=0b4b2ee27141432e8ca02dac223fb495)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:15.802929 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6283ecc-41d5-406c-8974-e520dfc17795 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:15.805277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 203/415] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:15 2022] GET /identity/v3/projects/0b4b2ee27141432e8ca02dac223fb495 => generated 344 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:15.811421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6283ecc-41d5-406c-8974-e520dfc17795 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:15.832820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 213/416] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 21:00:15.923606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:15.983059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6283ecc-41d5-406c-8974-e520dfc17795 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:16.029355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616c541-2ad3-40a8-b0c7-e0a8d7a2bffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:16.030030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616c541-2ad3-40a8-b0c7-e0a8d7a2bffa None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:16.030305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616c541-2ad3-40a8-b0c7-e0a8d7a2bffa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:16.030577 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616c541-2ad3-40a8-b0c7-e0a8d7a2bffa None admin] PATH_INFO: `/v3/projects/0b4b2ee27141432e8ca02dac223fb495/groups/e34a622ca2ef442b8d0162423cba4f2f/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:16.047283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616c541-2ad3-40a8-b0c7-e0a8d7a2bffa None admin] RBAC: Authorizing `identity:create_grant(project_id=0b4b2ee27141432e8ca02dac223fb495, group_id=e34a622ca2ef442b8d0162423cba4f2f, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:16.048802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616c541-2ad3-40a8-b0c7-e0a8d7a2bffa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:16.078956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 204/417] 10.222.0.79 () {58 vars in 1442 bytes} [Tue Sep 6 21:00:15 2022] PUT /identity/v3/projects/0b4b2ee27141432e8ca02dac223fb495/groups/e34a622ca2ef442b8d0162423cba4f2f/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:00:16.146433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc5485-21a1-485a-a491-e9a4aa1492fa None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:16.218495 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:16.261391 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc5485-21a1-485a-a491-e9a4aa1492fa None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:16.303331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a6d92-0cf6-4ee0-abda-58e33d0f09b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:16.303980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a6d92-0cf6-4ee0-abda-58e33d0f09b8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:16.304250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a6d92-0cf6-4ee0-abda-58e33d0f09b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:16.304518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a6d92-0cf6-4ee0-abda-58e33d0f09b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:16.316139 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a6d92-0cf6-4ee0-abda-58e33d0f09b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f85a6d92-0cf6-4ee0-abda-58e33d0f09b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:16.321458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:16.322696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:16.323806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:16.323806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:16.323806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:16.323806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:16.323806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:16.323806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:16.323806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:16.323806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:16.323806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:16.323806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:16.323806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:16.323806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:16.326893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 214/418] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:00:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:16.330366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616c541-2ad3-40a8-b0c7-e0a8d7a2bffa None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:16.437399 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616c541-2ad3-40a8-b0c7-e0a8d7a2bffa None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:16.463187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:16.485291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647dc29-9acb-4254-8196-b107801179d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:16.485829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647dc29-9acb-4254-8196-b107801179d6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:16.486106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647dc29-9acb-4254-8196-b107801179d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:16.486380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647dc29-9acb-4254-8196-b107801179d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:16.487076 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647dc29-9acb-4254-8196-b107801179d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:16.487358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647dc29-9acb-4254-8196-b107801179d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:16.488802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647dc29-9acb-4254-8196-b107801179d6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:16.500877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 205/419] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:16.506835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a6d92-0cf6-4ee0-abda-58e33d0f09b8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:16.684772 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a6d92-0cf6-4ee0-abda-58e33d0f09b8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:16.724360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:16.742819 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e383cf-7419-4b6e-a2ce-6b1d69ea6e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:16.743366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e383cf-7419-4b6e-a2ce-6b1d69ea6e3f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:16.743642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e383cf-7419-4b6e-a2ce-6b1d69ea6e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:16.743914 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e383cf-7419-4b6e-a2ce-6b1d69ea6e3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:16.794213 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9eb7cf-c85b-41eb-bd25-c438874f3808 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:16.794608 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9eb7cf-c85b-41eb-bd25-c438874f3808 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:16.794899 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9eb7cf-c85b-41eb-bd25-c438874f3808 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:16.796684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 206/420] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:16.801985 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5efe8-88e8-497b-a846-4f2c39a1d4c1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:16.802506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5efe8-88e8-497b-a846-4f2c39a1d4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:16.802848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5efe8-88e8-497b-a846-4f2c39a1d4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:16.837773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c4d5efe8-88e8-497b-a846-4f2c39a1d4c1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:16.838981 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c4d5efe8-88e8-497b-a846-4f2c39a1d4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:16.842956 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e383cf-7419-4b6e-a2ce-6b1d69ea6e3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:16.844426 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e383cf-7419-4b6e-a2ce-6b1d69ea6e3f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:16.957818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 207/421] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:17.017989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 215/422] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:17.023507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5efe8-88e8-497b-a846-4f2c39a1d4c1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:17.139971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5efe8-88e8-497b-a846-4f2c39a1d4c1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:17.149594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:17.175065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f0f5b-cb90-477b-b526-b2a13dd2ecfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:17.175714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f0f5b-cb90-477b-b526-b2a13dd2ecfa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:17.175989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f0f5b-cb90-477b-b526-b2a13dd2ecfa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:17.176258 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f0f5b-cb90-477b-b526-b2a13dd2ecfa None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:17.177395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f0f5b-cb90-477b-b526-b2a13dd2ecfa None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:17.178855 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f0f5b-cb90-477b-b526-b2a13dd2ecfa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:17.180817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 208/423] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:17 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:17.186667 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e383cf-7419-4b6e-a2ce-6b1d69ea6e3f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:17.231869 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:17.336987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e383cf-7419-4b6e-a2ce-6b1d69ea6e3f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:17.382700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f0f5b-cb90-477b-b526-b2a13dd2ecfa None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:17.384050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de092867-cce2-408d-89bb-04f203e1c3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:17.384757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de092867-cce2-408d-89bb-04f203e1c3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:17.385042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de092867-cce2-408d-89bb-04f203e1c3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:17.385311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de092867-cce2-408d-89bb-04f203e1c3e7 None admin] PATH_INFO: `/v3/groups/09b16cf2af00486cb37e47678e78fba2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:17.386455 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de092867-cce2-408d-89bb-04f203e1c3e7 None admin] RBAC: Authorizing `identity:get_group(group_id=09b16cf2af00486cb37e47678e78fba2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:17.387950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de092867-cce2-408d-89bb-04f203e1c3e7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:17.390896 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 216/424] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:17 2022] GET /identity/v3/groups/09b16cf2af00486cb37e47678e78fba2 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:17.395794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de092867-cce2-408d-89bb-04f203e1c3e7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:17.553876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:17.556096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f0f5b-cb90-477b-b526-b2a13dd2ecfa None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:17.574198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de092867-cce2-408d-89bb-04f203e1c3e7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:17.636968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaee19e-cd83-46e1-882e-5bdc98e32ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:17.637657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaee19e-cd83-46e1-882e-5bdc98e32ecf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:17.637947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaee19e-cd83-46e1-882e-5bdc98e32ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:17.638219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaee19e-cd83-46e1-882e-5bdc98e32ecf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:17.638961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaee19e-cd83-46e1-882e-5bdc98e32ecf None admin] RBAC: Adding query filter params (group.id=09b16cf2af00486cb37e47678e78fba2, role.id=e945f087a5bc4c38bd8848782f76c214, scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:17.639268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaee19e-cd83-46e1-882e-5bdc98e32ecf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:17.641042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaee19e-cd83-46e1-882e-5bdc98e32ecf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:17.649186 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 217/425] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Sep 6 21:00:17 2022] GET /identity/v3/role_assignments?group.id=09b16cf2af00486cb37e47678e78fba2&scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e&role.id=e945f087a5bc4c38bd8848782f76c214 => generated 611 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:17.650487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50844f-afb0-479f-a1cc-6526bd651d3a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:17.650774 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50844f-afb0-479f-a1cc-6526bd651d3a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:17.651056 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50844f-afb0-479f-a1cc-6526bd651d3a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:17.652609 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9c7ce-e112-4e44-b843-2c393e2f3433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:17.652801 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 218/426] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:17.653701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9c7ce-e112-4e44-b843-2c393e2f3433 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:17.653981 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9c7ce-e112-4e44-b843-2c393e2f3433 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:17.654250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9c7ce-e112-4e44-b843-2c393e2f3433 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:17.661600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641bcad-fb00-4a03-aff7-f1261406d9ee None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:17.662449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641bcad-fb00-4a03-aff7-f1261406d9ee None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:17.663123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641bcad-fb00-4a03-aff7-f1261406d9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:17.671514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9c7ce-e112-4e44-b843-2c393e2f3433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-14d9c7ce-e112-4e44-b843-2c393e2f3433 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:17.677512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:17.679221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:17.680707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:17.680707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:17.680707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:17.680707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:17.680707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:17.680707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:17.680707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:17.680707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:17.680707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:17.680707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:17.680707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:17.680707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:17.682177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 209/427] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:17.693361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9c7ce-e112-4e44-b843-2c393e2f3433 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:17.693926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8641bcad-fb00-4a03-aff7-f1261406d9ee None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:17.695550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8641bcad-fb00-4a03-aff7-f1261406d9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:17.784167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:17.795097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 219/428] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:17.918049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9c7ce-e112-4e44-b843-2c393e2f3433 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:17.965801 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487270c7-1acc-4ee9-a1b4-adce98ca2b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:17.966377 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487270c7-1acc-4ee9-a1b4-adce98ca2b21 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:17.966654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487270c7-1acc-4ee9-a1b4-adce98ca2b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:17.966925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487270c7-1acc-4ee9-a1b4-adce98ca2b21 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:17.967598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487270c7-1acc-4ee9-a1b4-adce98ca2b21 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:17.967896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487270c7-1acc-4ee9-a1b4-adce98ca2b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:17.969331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487270c7-1acc-4ee9-a1b4-adce98ca2b21 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:17.982314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 210/429] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 21:00:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:17.987950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641bcad-fb00-4a03-aff7-f1261406d9ee None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:18.093017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487270c7-1acc-4ee9-a1b4-adce98ca2b21 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:18.098941 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:18.115605 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641bcad-fb00-4a03-aff7-f1261406d9ee None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:18.172811 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b82d8-00f7-43e7-9e58-fb671310ad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:18.173364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b82d8-00f7-43e7-9e58-fb671310ad69 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:18.173643 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b82d8-00f7-43e7-9e58-fb671310ad69 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:18.173914 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b82d8-00f7-43e7-9e58-fb671310ad69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:18.240148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:18.248076 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487270c7-1acc-4ee9-a1b4-adce98ca2b21 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:18.254201 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b82d8-00f7-43e7-9e58-fb671310ad69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:18.255715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b82d8-00f7-43e7-9e58-fb671310ad69 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:18.290912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb9101-f1cb-4060-9d26-b17fa40aff69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:18.291558 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb9101-f1cb-4060-9d26-b17fa40aff69 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:18.291843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb9101-f1cb-4060-9d26-b17fa40aff69 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:18.292115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb9101-f1cb-4060-9d26-b17fa40aff69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:18.304521 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb9101-f1cb-4060-9d26-b17fa40aff69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-07cb9101-f1cb-4060-9d26-b17fa40aff69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:18.310038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:18.311243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:18.312046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:18.312046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:18.312046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:18.312046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:18.312046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:18.312046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:18.312046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:18.312046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:18.312046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:18.312046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:18.312046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 21:00:18.312046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:18.312046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 211/430] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:00:18.313402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32998a3c-1d7c-4aa2-a68c-4fd7ec090855 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:18.313402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32998a3c-1d7c-4aa2-a68c-4fd7ec090855 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:18.313402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32998a3c-1d7c-4aa2-a68c-4fd7ec090855 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:18.314619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 212/431] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:18.319479 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32998a3c-1d7c-4aa2-a68c-4fd7ec090855 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:18.362919 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:18.382983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 220/432] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:18.384438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c51fc-8818-4731-9ce6-312b64a391f8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:18.384745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c51fc-8818-4731-9ce6-312b64a391f8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:18.385018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c51fc-8818-4731-9ce6-312b64a391f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:18.403817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-355c51fc-8818-4731-9ce6-312b64a391f8 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:18.405086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-355c51fc-8818-4731-9ce6-312b64a391f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:18.444975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32998a3c-1d7c-4aa2-a68c-4fd7ec090855 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:18.483870 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f09db-77a8-4673-8b4b-56b4f09df9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:18.484473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f09db-77a8-4673-8b4b-56b4f09df9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:18.484775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f09db-77a8-4673-8b4b-56b4f09df9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:18.485042 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f09db-77a8-4673-8b4b-56b4f09df9d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:18.485314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 221/433] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:18.486300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c51fc-8818-4731-9ce6-312b64a391f8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:18.489098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f09db-77a8-4673-8b4b-56b4f09df9d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:18.490345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f09db-77a8-4673-8b4b-56b4f09df9d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:18.491921 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f09db-77a8-4673-8b4b-56b4f09df9d5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:18.507736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 213/434] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 21:00:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:18.513741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f09db-77a8-4673-8b4b-56b4f09df9d5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:18.543297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:18.636982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c51fc-8818-4731-9ce6-312b64a391f8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:18.660143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:18.669502 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f09db-77a8-4673-8b4b-56b4f09df9d5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:18.685359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c340a-1ea8-443f-842e-ecbdab549740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:18.686370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c340a-1ea8-443f-842e-ecbdab549740 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:18.686650 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c340a-1ea8-443f-842e-ecbdab549740 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:18.686922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c340a-1ea8-443f-842e-ecbdab549740 None admin] PATH_INFO: `/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:18.688372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c340a-1ea8-443f-842e-ecbdab549740 None admin] RBAC: Authorizing `identity:get_project(project_id=c83c3b8acaf54e5f8375853e5b4a7e1e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:18.689910 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c340a-1ea8-443f-842e-ecbdab549740 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:18.692813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 222/435] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:18 2022] GET /identity/v3/projects/c83c3b8acaf54e5f8375853e5b4a7e1e => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:18.701082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c340a-1ea8-443f-842e-ecbdab549740 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:18.727091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08b956-3a9a-4b84-aae8-66999dd5caea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:18.728817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08b956-3a9a-4b84-aae8-66999dd5caea None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:18.729609 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08b956-3a9a-4b84-aae8-66999dd5caea None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:18.730329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08b956-3a9a-4b84-aae8-66999dd5caea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:18.816280 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08b956-3a9a-4b84-aae8-66999dd5caea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:18.817806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08b956-3a9a-4b84-aae8-66999dd5caea None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:18.821775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c340a-1ea8-443f-842e-ecbdab549740 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:18.839960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:18.856241 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eeacb-55ec-486d-89d0-ebc105a653dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:18.856924 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eeacb-55ec-486d-89d0-ebc105a653dd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:18.857203 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eeacb-55ec-486d-89d0-ebc105a653dd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:18.857480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eeacb-55ec-486d-89d0-ebc105a653dd None admin] PATH_INFO: `/v3/groups/09b16cf2af00486cb37e47678e78fba2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:18.858647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2eeacb-55ec-486d-89d0-ebc105a653dd None admin] RBAC: Authorizing `identity:get_group(group_id=09b16cf2af00486cb37e47678e78fba2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:18.860098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2eeacb-55ec-486d-89d0-ebc105a653dd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:18.862007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 223/436] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:18 2022] GET /identity/v3/groups/09b16cf2af00486cb37e47678e78fba2 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:18.863001 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2c5e9-a80b-4d27-9c2e-2050b7c7477a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:18.863372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2c5e9-a80b-4d27-9c2e-2050b7c7477a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:18.863647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2c5e9-a80b-4d27-9c2e-2050b7c7477a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:18.865058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 224/437] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:18.865647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2c5e9-a80b-4d27-9c2e-2050b7c7477a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:18.919632 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:18.924410 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 214/438] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 21:00:18.925195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08b956-3a9a-4b84-aae8-66999dd5caea None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:18.981816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2c5e9-a80b-4d27-9c2e-2050b7c7477a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:19.030087 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baff0d-2cb1-4cbf-aabb-bca1a509023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:19.031270 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baff0d-2cb1-4cbf-aabb-bca1a509023a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:19.032553 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baff0d-2cb1-4cbf-aabb-bca1a509023a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:19.033021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baff0d-2cb1-4cbf-aabb-bca1a509023a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:19.040294 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08b956-3a9a-4b84-aae8-66999dd5caea None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:19.049459 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baff0d-2cb1-4cbf-aabb-bca1a509023a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e3baff0d-2cb1-4cbf-aabb-bca1a509023a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:19.055284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:19.061655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 225/439] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:00:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:19.063233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3876c-ded5-46be-999f-14b6e4af0471 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:19.064594 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3876c-ded5-46be-999f-14b6e4af0471 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:19.064594 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3876c-ded5-46be-999f-14b6e4af0471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:19.069054 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:19.078529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-92f3876c-ded5-46be-999f-14b6e4af0471 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:19.079745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-92f3876c-ded5-46be-999f-14b6e4af0471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:19.084930 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3aecd3-eeb3-4f13-8dcb-6faa376cfe86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:19.085546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3aecd3-eeb3-4f13-8dcb-6faa376cfe86 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:19.085831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3aecd3-eeb3-4f13-8dcb-6faa376cfe86 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:19.086104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3aecd3-eeb3-4f13-8dcb-6faa376cfe86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:19.086832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3aecd3-eeb3-4f13-8dcb-6faa376cfe86 None admin] RBAC: Adding query filter params (group.id=09b16cf2af00486cb37e47678e78fba2, role.id=180edbb8a9e74555af89d6a7639f819a, scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:19.087139 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3aecd3-eeb3-4f13-8dcb-6faa376cfe86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:19.088588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3aecd3-eeb3-4f13-8dcb-6faa376cfe86 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:19.094387 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 215/440] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Sep 6 21:00:18 2022] GET /identity/v3/role_assignments?group.id=09b16cf2af00486cb37e47678e78fba2&scope.project.id=c83c3b8acaf54e5f8375853e5b4a7e1e&role.id=180edbb8a9e74555af89d6a7639f819a => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:19.095136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3aecd3-eeb3-4f13-8dcb-6faa376cfe86 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:19.164827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 226/441] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:19.166028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3876c-ded5-46be-999f-14b6e4af0471 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:19.216166 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:19.245430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3aecd3-eeb3-4f13-8dcb-6faa376cfe86 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:19.285867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164ede1-11cc-4f70-8c21-cebe3d86b8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:19.286686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164ede1-11cc-4f70-8c21-cebe3d86b8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:19.287066 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164ede1-11cc-4f70-8c21-cebe3d86b8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:19.287477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164ede1-11cc-4f70-8c21-cebe3d86b8d9 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:19.289417 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164ede1-11cc-4f70-8c21-cebe3d86b8d9 None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:19.289901 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3876c-ded5-46be-999f-14b6e4af0471 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:19.290880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164ede1-11cc-4f70-8c21-cebe3d86b8d9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:19.292768 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 216/442] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:19 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:19.299162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164ede1-11cc-4f70-8c21-cebe3d86b8d9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:19.343810 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500dbf6c-4ea0-4753-a60a-55255692df87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:19.344726 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dbf6c-4ea0-4753-a60a-55255692df87 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:19.345132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dbf6c-4ea0-4753-a60a-55255692df87 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:19.345523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dbf6c-4ea0-4753-a60a-55255692df87 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:19.346580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500dbf6c-4ea0-4753-a60a-55255692df87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:19.347022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500dbf6c-4ea0-4753-a60a-55255692df87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:19.349147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500dbf6c-4ea0-4753-a60a-55255692df87 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:19.366566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 227/443] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:19.367296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:19.368769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500dbf6c-4ea0-4753-a60a-55255692df87 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:19.431358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164ede1-11cc-4f70-8c21-cebe3d86b8d9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:19.471613 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b31f9-0066-4074-a155-ba16a472dbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:19.472991 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b31f9-0066-4074-a155-ba16a472dbe0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:19.473283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b31f9-0066-4074-a155-ba16a472dbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:19.473549 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b31f9-0066-4074-a155-ba16a472dbe0 None admin] PATH_INFO: `/v3/groups/09b16cf2af00486cb37e47678e78fba2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:19.475826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b31f9-0066-4074-a155-ba16a472dbe0 None admin] RBAC: Authorizing `identity:get_group(group_id=09b16cf2af00486cb37e47678e78fba2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:19.476203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b31f9-0066-4074-a155-ba16a472dbe0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:19.477838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 217/444] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:19 2022] GET /identity/v3/groups/09b16cf2af00486cb37e47678e78fba2 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:19.478648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b31f9-0066-4074-a155-ba16a472dbe0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:19.499704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500dbf6c-4ea0-4753-a60a-55255692df87 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:19.502706 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:19.539463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a987e-34a1-46c2-8688-4c0c37b5efe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:19.540316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a987e-34a1-46c2-8688-4c0c37b5efe3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:19.540799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a987e-34a1-46c2-8688-4c0c37b5efe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:19.541217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a987e-34a1-46c2-8688-4c0c37b5efe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:19.604520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b31f9-0066-4074-a155-ba16a472dbe0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:19.627912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:19.629639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a987e-34a1-46c2-8688-4c0c37b5efe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:19.631141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a987e-34a1-46c2-8688-4c0c37b5efe3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:19.643808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509f984-7a91-43e8-b9ea-130a1e25cf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:19.644451 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509f984-7a91-43e8-b9ea-130a1e25cf5f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:19.644760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509f984-7a91-43e8-b9ea-130a1e25cf5f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:19.645033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509f984-7a91-43e8-b9ea-130a1e25cf5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:19.657512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509f984-7a91-43e8-b9ea-130a1e25cf5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6509f984-7a91-43e8-b9ea-130a1e25cf5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:19.662864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:19.663727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:19.664504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:19.664504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:19.664504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:19.664504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:19.664504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:19.664504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:19.664504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:19.664504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:19.664504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:19.664504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:19.664504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:00:19.664504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:19.664504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 218/445] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:00:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:19.665804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509f984-7a91-43e8-b9ea-130a1e25cf5f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:19.749398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 228/446] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:19.750589 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a987e-34a1-46c2-8688-4c0c37b5efe3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:19.755732 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:19.777939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509f984-7a91-43e8-b9ea-130a1e25cf5f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:19.813281 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3828c9-fa02-4201-9cb0-df6b23a2b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:19.813935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3828c9-fa02-4201-9cb0-df6b23a2b626 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:19.814213 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3828c9-fa02-4201-9cb0-df6b23a2b626 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:19.814485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3828c9-fa02-4201-9cb0-df6b23a2b626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:19.815254 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3828c9-fa02-4201-9cb0-df6b23a2b626 None admin] RBAC: Adding query filter params (group.id=09b16cf2af00486cb37e47678e78fba2, role.id=180edbb8a9e74555af89d6a7639f819a, scope.project.id=402258177cb849268ec3155d32bedd84) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:19.815573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3828c9-fa02-4201-9cb0-df6b23a2b626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:19.817110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3828c9-fa02-4201-9cb0-df6b23a2b626 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:19.823330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 219/447] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Sep 6 21:00:19 2022] GET /identity/v3/role_assignments?group.id=09b16cf2af00486cb37e47678e78fba2&scope.project.id=402258177cb849268ec3155d32bedd84&role.id=180edbb8a9e74555af89d6a7639f819a => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:19.824119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3828c9-fa02-4201-9cb0-df6b23a2b626 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:19.858727 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a987e-34a1-46c2-8688-4c0c37b5efe3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:19.885586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:19.902366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25748b5-973c-4925-a784-3a1ff91d61c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:19.903011 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25748b5-973c-4925-a784-3a1ff91d61c3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:19.903314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25748b5-973c-4925-a784-3a1ff91d61c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:19.903584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25748b5-973c-4925-a784-3a1ff91d61c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:19.904296 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25748b5-973c-4925-a784-3a1ff91d61c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:19.904597 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25748b5-973c-4925-a784-3a1ff91d61c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:19.906128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25748b5-973c-4925-a784-3a1ff91d61c3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:19.920124 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 229/448] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:19.926337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25748b5-973c-4925-a784-3a1ff91d61c3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:19.955188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3828c9-fa02-4201-9cb0-df6b23a2b626 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:19.999153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3930ef-7f3f-4947-9157-793ee6170188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:19.999805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3930ef-7f3f-4947-9157-793ee6170188 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:20.000085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3930ef-7f3f-4947-9157-793ee6170188 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:20.000350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3930ef-7f3f-4947-9157-793ee6170188 None admin] PATH_INFO: `/v3/projects/402258177cb849268ec3155d32bedd84` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:20.001656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3930ef-7f3f-4947-9157-793ee6170188 None admin] RBAC: Authorizing `identity:get_project(project_id=402258177cb849268ec3155d32bedd84)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:20.003083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3930ef-7f3f-4947-9157-793ee6170188 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:20.004496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:20.009837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 220/449] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:19 2022] GET /identity/v3/projects/402258177cb849268ec3155d32bedd84 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:20.011300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3930ef-7f3f-4947-9157-793ee6170188 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:20.050244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25748b5-973c-4925-a784-3a1ff91d61c3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:20.094153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41395d2c-a597-409b-9777-cead4ec8ea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:20.094771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41395d2c-a597-409b-9777-cead4ec8ea90 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:20.095042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41395d2c-a597-409b-9777-cead4ec8ea90 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:20.095305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41395d2c-a597-409b-9777-cead4ec8ea90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:20.128352 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3930ef-7f3f-4947-9157-793ee6170188 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:20.143402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:20.169762 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bee6ea-f463-4d7c-be55-dea24b5aa4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:20.170722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bee6ea-f463-4d7c-be55-dea24b5aa4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:20.170999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bee6ea-f463-4d7c-be55-dea24b5aa4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:20.171266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bee6ea-f463-4d7c-be55-dea24b5aa4a4 None admin] PATH_INFO: `/v3/groups/09b16cf2af00486cb37e47678e78fba2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:20.172396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bee6ea-f463-4d7c-be55-dea24b5aa4a4 None admin] RBAC: Authorizing `identity:get_group(group_id=09b16cf2af00486cb37e47678e78fba2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:20.173909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bee6ea-f463-4d7c-be55-dea24b5aa4a4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:20.175664 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 221/450] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:20 2022] GET /identity/v3/groups/09b16cf2af00486cb37e47678e78fba2 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:20.181909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bee6ea-f463-4d7c-be55-dea24b5aa4a4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:20.191783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41395d2c-a597-409b-9777-cead4ec8ea90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:20.193647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41395d2c-a597-409b-9777-cead4ec8ea90 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:20.262442 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:20.284755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bee6ea-f463-4d7c-be55-dea24b5aa4a4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:20.298489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 230/451] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 21:00:20.304315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41395d2c-a597-409b-9777-cead4ec8ea90 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:20.317976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57deded-b641-4ed1-902c-4e901684802c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:20.318589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57deded-b641-4ed1-902c-4e901684802c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:20.318871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57deded-b641-4ed1-902c-4e901684802c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:20.319133 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57deded-b641-4ed1-902c-4e901684802c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:20.319861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57deded-b641-4ed1-902c-4e901684802c None admin] RBAC: Adding query filter params (group.id=09b16cf2af00486cb37e47678e78fba2, role.id=e945f087a5bc4c38bd8848782f76c214, scope.project.id=402258177cb849268ec3155d32bedd84) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:20.320163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57deded-b641-4ed1-902c-4e901684802c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:20.321650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57deded-b641-4ed1-902c-4e901684802c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:20.327349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 222/452] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Sep 6 21:00:20 2022] GET /identity/v3/role_assignments?group.id=09b16cf2af00486cb37e47678e78fba2&scope.project.id=402258177cb849268ec3155d32bedd84&role.id=e945f087a5bc4c38bd8848782f76c214 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:20.414375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41395d2c-a597-409b-9777-cead4ec8ea90 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:20.453478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e66ef-45ec-49bc-ba90-4e327ea55e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:20.454126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e66ef-45ec-49bc-ba90-4e327ea55e6e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:20.454403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e66ef-45ec-49bc-ba90-4e327ea55e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:20.454670 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e66ef-45ec-49bc-ba90-4e327ea55e6e None admin] PATH_INFO: `/v3/projects/0b4b2ee27141432e8ca02dac223fb495` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:20.455872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e66ef-45ec-49bc-ba90-4e327ea55e6e None admin] RBAC: Authorizing `identity:get_project(project_id=0b4b2ee27141432e8ca02dac223fb495)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:20.457383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e66ef-45ec-49bc-ba90-4e327ea55e6e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:20.459114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 231/453] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:00:20 2022] GET /identity/v3/projects/0b4b2ee27141432e8ca02dac223fb495 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:20.464964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57deded-b641-4ed1-902c-4e901684802c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:20.517194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:20.578049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57deded-b641-4ed1-902c-4e901684802c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:20.616151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7bf42-678e-4c0a-b6b5-351d02271f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:20.616809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7bf42-678e-4c0a-b6b5-351d02271f07 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:20.617093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7bf42-678e-4c0a-b6b5-351d02271f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:20.617363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7bf42-678e-4c0a-b6b5-351d02271f07 None admin] PATH_INFO: `/v3/groups/e34a622ca2ef442b8d0162423cba4f2f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:20.618487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7bf42-678e-4c0a-b6b5-351d02271f07 None admin] RBAC: Authorizing `identity:get_group(group_id=e34a622ca2ef442b8d0162423cba4f2f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:20.619941 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7bf42-678e-4c0a-b6b5-351d02271f07 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:20.621571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 223/454] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Sep 6 21:00:20 2022] GET /identity/v3/groups/e34a622ca2ef442b8d0162423cba4f2f => generated 229 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:20.627556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e66ef-45ec-49bc-ba90-4e327ea55e6e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:20.664880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:20.734469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e66ef-45ec-49bc-ba90-4e327ea55e6e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:20.770962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb8b07-e245-46b8-a64c-88f1f238142b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:20.771478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb8b07-e245-46b8-a64c-88f1f238142b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:20.771752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb8b07-e245-46b8-a64c-88f1f238142b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:20.772002 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb8b07-e245-46b8-a64c-88f1f238142b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:20.772746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb8b07-e245-46b8-a64c-88f1f238142b None admin] RBAC: Adding query filter params (group.id=e34a622ca2ef442b8d0162423cba4f2f, role.id=3b30504991194002b41f8587d0851c10, scope.project.id=0b4b2ee27141432e8ca02dac223fb495) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:20.773054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb8b07-e245-46b8-a64c-88f1f238142b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:20.774474 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb8b07-e245-46b8-a64c-88f1f238142b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:20.781222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 232/455] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Sep 6 21:00:20 2022] GET /identity/v3/role_assignments?group.id=e34a622ca2ef442b8d0162423cba4f2f&scope.project.id=0b4b2ee27141432e8ca02dac223fb495&role.id=3b30504991194002b41f8587d0851c10 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:23.475364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a9212-bb0e-4155-bfdc-dc0a67a51a96 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:23.476389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a9212-bb0e-4155-bfdc-dc0a67a51a96 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:23.476901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a9212-bb0e-4155-bfdc-dc0a67a51a96 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:23.478766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 224/456] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:23.484057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235ddba-9ab7-4e5c-88e9-3cbb11fd79d9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:23.484959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235ddba-9ab7-4e5c-88e9-3cbb11fd79d9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:23.485514 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235ddba-9ab7-4e5c-88e9-3cbb11fd79d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:23.507697 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7235ddba-9ab7-4e5c-88e9-3cbb11fd79d9 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:23.509317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7235ddba-9ab7-4e5c-88e9-3cbb11fd79d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:23.569797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b20eb2-0b53-4710-8cc2-6e2632137e0c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:23.570466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b20eb2-0b53-4710-8cc2-6e2632137e0c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:23.570933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b20eb2-0b53-4710-8cc2-6e2632137e0c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:23.572809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 225/457] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:23.577831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7663d3-98e8-4c03-b1a9-329ea039362e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:23.578361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7663d3-98e8-4c03-b1a9-329ea039362e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:23.578851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7663d3-98e8-4c03-b1a9-329ea039362e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:23.608607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 233/458] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:23.616782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ad7663d3-98e8-4c03-b1a9-329ea039362e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:23.618007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ad7663d3-98e8-4c03-b1a9-329ea039362e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:23.713652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 226/459] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:23.924424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9814af-b085-430a-85de-70a61f40bd28 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:23.924849 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9814af-b085-430a-85de-70a61f40bd28 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:23.925027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9814af-b085-430a-85de-70a61f40bd28 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:23.926764 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 234/460] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:23.945719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca70d5-083b-4f30-af6f-0359db71ac62 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:23.946064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca70d5-083b-4f30-af6f-0359db71ac62 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:23.946267 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca70d5-083b-4f30-af6f-0359db71ac62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:23.964426 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9814af-b085-430a-85de-70a61f40bd28 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:23.970089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-daca70d5-083b-4f30-af6f-0359db71ac62 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:23.972368 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-daca70d5-083b-4f30-af6f-0359db71ac62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:24.009963 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:24.017907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:24.059184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 227/461] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:24.060407 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca70d5-083b-4f30-af6f-0359db71ac62 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:24.104794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9814af-b085-430a-85de-70a61f40bd28 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:24.141124 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643eb0f-334d-4fcb-a1d2-19629a86ae82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:24.141808 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643eb0f-334d-4fcb-a1d2-19629a86ae82 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:24.142086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643eb0f-334d-4fcb-a1d2-19629a86ae82 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:24.142355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643eb0f-334d-4fcb-a1d2-19629a86ae82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:24.155313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643eb0f-334d-4fcb-a1d2-19629a86ae82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:24.155313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643eb0f-334d-4fcb-a1d2-19629a86ae82 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9643eb0f-334d-4fcb-a1d2-19629a86ae82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:24.172638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:24.173571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:24.174403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:24.174403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:24.174403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:24.179250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 235/462] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:24.180599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936ab95-4d3b-470a-b835-68c5736b8b04 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:24.181042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936ab95-4d3b-470a-b835-68c5736b8b04 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:24.181366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936ab95-4d3b-470a-b835-68c5736b8b04 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:24.182736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca70d5-083b-4f30-af6f-0359db71ac62 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:24.182915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 236/463] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:24.183610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936ab95-4d3b-470a-b835-68c5736b8b04 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:24.221285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b4c09-919a-4dd6-80ad-06863355b658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:24.221930 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b4c09-919a-4dd6-80ad-06863355b658 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:24.222483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b4c09-919a-4dd6-80ad-06863355b658 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:24.222700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b4c09-919a-4dd6-80ad-06863355b658 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:24.239222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b4c09-919a-4dd6-80ad-06863355b658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:24.241725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b4c09-919a-4dd6-80ad-06863355b658 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ed8b4c09-919a-4dd6-80ad-06863355b658 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:24.266481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:24.267423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:24.268302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:24.268302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:24.268302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:24.268302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 228/464] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:24.274409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf427ff-d8e4-4c0c-9fd6-48672daa013c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:24.274760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf427ff-d8e4-4c0c-9fd6-48672daa013c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:24.274947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf427ff-d8e4-4c0c-9fd6-48672daa013c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:24.313334 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:24.315759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-dbf427ff-d8e4-4c0c-9fd6-48672daa013c None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:24.317005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dbf427ff-d8e4-4c0c-9fd6-48672daa013c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:24.351806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936ab95-4d3b-470a-b835-68c5736b8b04 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:24.353395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:24.391381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19d93a-84aa-423f-892e-c7f8c5053d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:24.392000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19d93a-84aa-423f-892e-c7f8c5053d4f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:24.392279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19d93a-84aa-423f-892e-c7f8c5053d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:24.392547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19d93a-84aa-423f-892e-c7f8c5053d4f None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:24.393498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19d93a-84aa-423f-892e-c7f8c5053d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:24.393803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19d93a-84aa-423f-892e-c7f8c5053d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:24.395429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19d93a-84aa-423f-892e-c7f8c5053d4f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:24.408056 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 237/465] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:24.408056 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19d93a-84aa-423f-892e-c7f8c5053d4f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:24.428743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 229/466] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:24.428743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf427ff-d8e4-4c0c-9fd6-48672daa013c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:24.549535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19d93a-84aa-423f-892e-c7f8c5053d4f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:24.576056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf427ff-d8e4-4c0c-9fd6-48672daa013c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:24.627331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cda09-f7bc-4c36-8044-12ea98dba112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:24.628132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cda09-f7bc-4c36-8044-12ea98dba112 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:24.628481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cda09-f7bc-4c36-8044-12ea98dba112 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:24.628682 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cda09-f7bc-4c36-8044-12ea98dba112 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:24.629369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cda09-f7bc-4c36-8044-12ea98dba112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:24.629680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cda09-f7bc-4c36-8044-12ea98dba112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:24.631062 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cda09-f7bc-4c36-8044-12ea98dba112 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:24.632763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4f1e6-b528-4edf-8213-31e4fb3eb442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:24.633645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4f1e6-b528-4edf-8213-31e4fb3eb442 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:24.634173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4f1e6-b528-4edf-8213-31e4fb3eb442 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:24.634635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4f1e6-b528-4edf-8213-31e4fb3eb442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:24.647941 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4f1e6-b528-4edf-8213-31e4fb3eb442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:24.649880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4f1e6-b528-4edf-8213-31e4fb3eb442 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:24.658579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:24.669926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 238/467] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-90e4f1e6-b528-4edf-8213-31e4fb3eb442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:24.670882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:24.671732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:24.672542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:24.672542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:24.672542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:24.672773 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cda09-f7bc-4c36-8044-12ea98dba112 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:24.674622 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 230/468] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:24.675651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4f1e6-b528-4edf-8213-31e4fb3eb442 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:24.691980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:24.817082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cda09-f7bc-4c36-8044-12ea98dba112 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:24.824016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4f1e6-b528-4edf-8213-31e4fb3eb442 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:24.858096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368858b-2b2b-4522-a1ca-a4a295139854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:24.859087 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368858b-2b2b-4522-a1ca-a4a295139854 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:24.859536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368858b-2b2b-4522-a1ca-a4a295139854 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:24.860000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368858b-2b2b-4522-a1ca-a4a295139854 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:24.860847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368858b-2b2b-4522-a1ca-a4a295139854 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:24.861800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6af824-c561-44ee-a4ee-b40202c1e0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:24.862514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6af824-c561-44ee-a4ee-b40202c1e0d0 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:24.862794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6af824-c561-44ee-a4ee-b40202c1e0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:24.863063 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6af824-c561-44ee-a4ee-b40202c1e0d0 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:24.863665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368858b-2b2b-4522-a1ca-a4a295139854 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:24.863766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6af824-c561-44ee-a4ee-b40202c1e0d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:24.865365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6af824-c561-44ee-a4ee-b40202c1e0d0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:24.889687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 239/469] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 21:00:24 2022] POST /identity/v3/users => generated 246 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:00:24.890879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368858b-2b2b-4522-a1ca-a4a295139854 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:24.901167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 231/470] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 21:00:24 2022] POST /identity/v3/users => generated 248 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:00:24.901933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6af824-c561-44ee-a4ee-b40202c1e0d0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:25.007462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:25.070972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:25.124338 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368858b-2b2b-4522-a1ca-a4a295139854 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:25.139096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6af824-c561-44ee-a4ee-b40202c1e0d0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:25.175250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8dcf7-21d6-4ed4-9b09-b10f41f06058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:25.175872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8dcf7-21d6-4ed4-9b09-b10f41f06058 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:25.176150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8dcf7-21d6-4ed4-9b09-b10f41f06058 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:25.176431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8dcf7-21d6-4ed4-9b09-b10f41f06058 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:25.177221 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8dcf7-21d6-4ed4-9b09-b10f41f06058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:25.177524 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8dcf7-21d6-4ed4-9b09-b10f41f06058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:25.179112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8dcf7-21d6-4ed4-9b09-b10f41f06058 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:25.179975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f790c-6865-4f61-bdf4-a947cd983879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:25.180627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f790c-6865-4f61-bdf4-a947cd983879 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:25.180954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f790c-6865-4f61-bdf4-a947cd983879 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:25.183235 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f790c-6865-4f61-bdf4-a947cd983879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:25.194196 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 240/471] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:25.196458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f790c-6865-4f61-bdf4-a947cd983879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:25.197994 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f790c-6865-4f61-bdf4-a947cd983879 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:25.200774 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8dcf7-21d6-4ed4-9b09-b10f41f06058 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-939f790c-6865-4f61-bdf4-a947cd983879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:25.214478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:25.215415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:25.216359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:25.216359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:25.216359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:25.216359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 232/472] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:25.224787 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f790c-6865-4f61-bdf4-a947cd983879 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:25.323401 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8dcf7-21d6-4ed4-9b09-b10f41f06058 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:25.336473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f790c-6865-4f61-bdf4-a947cd983879 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:25.341651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:25.351938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:25.362311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713966d5-c285-4304-865a-0b3c7d641aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:25.363194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713966d5-c285-4304-865a-0b3c7d641aaa None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:25.363664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713966d5-c285-4304-865a-0b3c7d641aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:25.364070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713966d5-c285-4304-865a-0b3c7d641aaa None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:25.364940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713966d5-c285-4304-865a-0b3c7d641aaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:25.366557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713966d5-c285-4304-865a-0b3c7d641aaa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:25.374121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53e5a9-437b-4700-8aee-7d08363cc831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:25.374867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53e5a9-437b-4700-8aee-7d08363cc831 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:25.375244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53e5a9-437b-4700-8aee-7d08363cc831 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:25.375680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53e5a9-437b-4700-8aee-7d08363cc831 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:25.376442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53e5a9-437b-4700-8aee-7d08363cc831 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:25.376987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53e5a9-437b-4700-8aee-7d08363cc831 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:25.379154 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53e5a9-437b-4700-8aee-7d08363cc831 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:25.395595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 241/473] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 21:00:25 2022] POST /identity/v3/users => generated 248 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:00:25.400766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 233/474] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:25.401421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6d6b5-7121-4f56-9972-7baa2b2c93b3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:25.401934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6d6b5-7121-4f56-9972-7baa2b2c93b3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:25.402330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6d6b5-7121-4f56-9972-7baa2b2c93b3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:25.404328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 242/475] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:25.408681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53e5a9-437b-4700-8aee-7d08363cc831 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:25.414256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651fe7a-dd1b-4af9-b6e9-fe372a2cf035 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:25.414545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651fe7a-dd1b-4af9-b6e9-fe372a2cf035 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:25.414807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651fe7a-dd1b-4af9-b6e9-fe372a2cf035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:25.461862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4651fe7a-dd1b-4af9-b6e9-fe372a2cf035 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:25.463125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4651fe7a-dd1b-4af9-b6e9-fe372a2cf035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:25.570452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53e5a9-437b-4700-8aee-7d08363cc831 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:25.610386 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c017896-bca9-4b73-9173-5d18d9e94957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:25.611027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 243/476] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:25.611376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c017896-bca9-4b73-9173-5d18d9e94957 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:25.611376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c017896-bca9-4b73-9173-5d18d9e94957 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:25.611681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c017896-bca9-4b73-9173-5d18d9e94957 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:25.612334 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c017896-bca9-4b73-9173-5d18d9e94957 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:25.613914 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c017896-bca9-4b73-9173-5d18d9e94957 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:25.645366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 234/477] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 21:00:25 2022] POST /identity/v3/users => generated 249 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:00:25.891310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651fe7a-dd1b-4af9-b6e9-fe372a2cf035 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:26.005953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651fe7a-dd1b-4af9-b6e9-fe372a2cf035 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:26.007750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:26.044302 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05aeae2-cc7d-4ae5-8f94-9f118ce18b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:26.044951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05aeae2-cc7d-4ae5-8f94-9f118ce18b7e None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:26.045226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05aeae2-cc7d-4ae5-8f94-9f118ce18b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:26.045491 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05aeae2-cc7d-4ae5-8f94-9f118ce18b7e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:26.046177 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05aeae2-cc7d-4ae5-8f94-9f118ce18b7e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:26.047568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05aeae2-cc7d-4ae5-8f94-9f118ce18b7e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:26.059272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.sql.core [None req-e05aeae2-cc7d-4ae5-8f94-9f118ce18b7e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 21:00:26.059272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 21:00:26.059272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [parameters: {'id': '4d3b17e8d3944f728e14e6f60872d898', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 21:00:26.059272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81697) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e05aeae2-cc7d-4ae5-8f94-9f118ce18b7e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Sep 06 21:00:26.078039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return ref.to_dict() Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 21:00:26.078942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application yield resource Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise self.value Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application session.commit() Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 21:00:26.079909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.flush() Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application compat.raise_( Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 21:00:26.080783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 21:00:26.081664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 21:00:26.082478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [parameters: {'id': '4d3b17e8d3944f728e14e6f60872d898', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:26.083227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 21:00:26.084020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:26.088239 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 244/478] 10.222.0.79 () {60 vars in 1131 bytes} [Tue Sep 6 21:00:25 2022] POST /identity/v3/roles => generated 144 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:26.088374 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c017896-bca9-4b73-9173-5d18d9e94957 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:26.126718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:26.220866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c017896-bca9-4b73-9173-5d18d9e94957 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:26.266393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ae15e-8c50-42f0-a200-7a4cb85d63e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:26.267047 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ae15e-8c50-42f0-a200-7a4cb85d63e0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:26.267324 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ae15e-8c50-42f0-a200-7a4cb85d63e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:26.267660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ae15e-8c50-42f0-a200-7a4cb85d63e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:26.279772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ae15e-8c50-42f0-a200-7a4cb85d63e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4e1ae15e-8c50-42f0-a200-7a4cb85d63e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:26.284746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:26.286662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:26.287742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:26.287742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:26.287742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:26.287742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:26.287742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:26.287742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:26.287742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:26.287742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:26.287742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:26.287742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:26.287742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 21:00:26.287742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:26.287742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 235/479] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:00:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:26.293455 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05aeae2-cc7d-4ae5-8f94-9f118ce18b7e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:26.406443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05aeae2-cc7d-4ae5-8f94-9f118ce18b7e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:26.441450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba8356-a23b-4db1-8989-4fe657dda59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:26.442075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba8356-a23b-4db1-8989-4fe657dda59e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:26.442353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba8356-a23b-4db1-8989-4fe657dda59e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:26.442624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba8356-a23b-4db1-8989-4fe657dda59e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:26.443291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba8356-a23b-4db1-8989-4fe657dda59e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:26.443591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba8356-a23b-4db1-8989-4fe657dda59e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:26.444996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba8356-a23b-4db1-8989-4fe657dda59e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:26.453604 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 245/480] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:27.539747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bea86-2688-4c6a-8325-404ce7d5d1a5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:27.540464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bea86-2688-4c6a-8325-404ce7d5d1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:27.540464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bea86-2688-4c6a-8325-404ce7d5d1a5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:27.542229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 236/481] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:27.543864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0b16c-c453-41a4-b96f-67c7abcc5786 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:27.544332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0b16c-c453-41a4-b96f-67c7abcc5786 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:27.544484 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0b16c-c453-41a4-b96f-67c7abcc5786 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:27.545870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 246/482] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:27.550872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8031e05-fafa-4aab-998b-ed8e212c1909 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:27.551194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8031e05-fafa-4aab-998b-ed8e212c1909 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:27.551501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8031e05-fafa-4aab-998b-ed8e212c1909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:27.561320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bcc83-431a-4b72-9b85-6724e012c659 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:27.561320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bcc83-431a-4b72-9b85-6724e012c659 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:27.561320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bcc83-431a-4b72-9b85-6724e012c659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:27.593452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f8031e05-fafa-4aab-998b-ed8e212c1909 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:27.595072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f8031e05-fafa-4aab-998b-ed8e212c1909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:27.601252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-549bcc83-431a-4b72-9b85-6724e012c659 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:27.603431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-549bcc83-431a-4b72-9b85-6724e012c659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:27.643181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:27.728512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 237/483] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:27.730011 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 247/484] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 21:00:28.001922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8031e05-fafa-4aab-998b-ed8e212c1909 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:28.025639 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:28.128496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bcc83-431a-4b72-9b85-6724e012c659 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:28.130687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8031e05-fafa-4aab-998b-ed8e212c1909 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:28.179526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd6377-7f31-4e30-bb3f-4fbf15201740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:28.180183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd6377-7f31-4e30-bb3f-4fbf15201740 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:28.180461 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd6377-7f31-4e30-bb3f-4fbf15201740 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:28.180881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd6377-7f31-4e30-bb3f-4fbf15201740 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:28.202430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd6377-7f31-4e30-bb3f-4fbf15201740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-78fd6377-7f31-4e30-bb3f-4fbf15201740 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:28.215138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:28.216201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:28.217034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:28.217034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:28.217034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:28.217034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:28.217034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:28.217034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:28.217034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:28.217034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:28.217034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:28.217034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:28.217034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:28.217034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:28.217034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 238/485] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:28.224350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd6377-7f31-4e30-bb3f-4fbf15201740 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:28.296760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bcc83-431a-4b72-9b85-6724e012c659 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:28.364081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37cd03-7b5c-4208-b5b7-750e06ad9266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:28.364765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37cd03-7b5c-4208-b5b7-750e06ad9266 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:28.365046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37cd03-7b5c-4208-b5b7-750e06ad9266 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:28.365317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37cd03-7b5c-4208-b5b7-750e06ad9266 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:28.379658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37cd03-7b5c-4208-b5b7-750e06ad9266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-de37cd03-7b5c-4208-b5b7-750e06ad9266 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:28.386128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:28.387050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:28.387926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:28.387926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:28.387926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:28.387926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:28.387926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:28.387926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:28.387926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:28.387926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:28.387926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:28.387926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:28.387926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:28.387926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:28.387926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 248/486] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:28.389967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:28.392985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6a90f-cda3-4cd8-bf00-20f545bd0709 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:28.393589 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6a90f-cda3-4cd8-bf00-20f545bd0709 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:28.397539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6a90f-cda3-4cd8-bf00-20f545bd0709 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:28.399654 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 249/487] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:28 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:28.400609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6a90f-cda3-4cd8-bf00-20f545bd0709 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:28.411206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd6377-7f31-4e30-bb3f-4fbf15201740 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:28.426300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:28.446950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135dce7-2607-4fe2-bc45-424e15dc8df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:28.447567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135dce7-2607-4fe2-bc45-424e15dc8df1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:28.447841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135dce7-2607-4fe2-bc45-424e15dc8df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:28.448101 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135dce7-2607-4fe2-bc45-424e15dc8df1 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:28.448805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135dce7-2607-4fe2-bc45-424e15dc8df1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:28.449100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135dce7-2607-4fe2-bc45-424e15dc8df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:28.450504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135dce7-2607-4fe2-bc45-424e15dc8df1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:28.458913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 239/488] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:28.459936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe1f3d-b82b-4d28-be69-ae62aff0a9c3 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:28.460218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe1f3d-b82b-4d28-be69-ae62aff0a9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:28.460488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe1f3d-b82b-4d28-be69-ae62aff0a9c3 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:28.461978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 240/489] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:28.463355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d43f6-ef5a-4107-bc49-6d849d63b6cf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:28.463626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d43f6-ef5a-4107-bc49-6d849d63b6cf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:28.463898 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d43f6-ef5a-4107-bc49-6d849d63b6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:28.482579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e24d43f6-ef5a-4107-bc49-6d849d63b6cf None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:28.483767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e24d43f6-ef5a-4107-bc49-6d849d63b6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:28.520546 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6a90f-cda3-4cd8-bf00-20f545bd0709 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:28.557899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a460507-8b70-4072-9ccc-fd1441aada2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:28.558780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a460507-8b70-4072-9ccc-fd1441aada2c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:28.558780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a460507-8b70-4072-9ccc-fd1441aada2c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:28.559149 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a460507-8b70-4072-9ccc-fd1441aada2c None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:28.560021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a460507-8b70-4072-9ccc-fd1441aada2c None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:28.560331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a460507-8b70-4072-9ccc-fd1441aada2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:28.568738 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a460507-8b70-4072-9ccc-fd1441aada2c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:28.568895 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 241/490] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:28.568895 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d43f6-ef5a-4107-bc49-6d849d63b6cf None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:28.581243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 250/491] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:28.581243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18154b05-59b4-4ca0-b81e-d8fff5a9a9aa None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:28.581243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18154b05-59b4-4ca0-b81e-d8fff5a9a9aa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:28.581243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18154b05-59b4-4ca0-b81e-d8fff5a9a9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:28.602607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-18154b05-59b4-4ca0-b81e-d8fff5a9a9aa None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:28.603930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-18154b05-59b4-4ca0-b81e-d8fff5a9a9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:28.657606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:28.674792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 251/492] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:28.675726 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18154b05-59b4-4ca0-b81e-d8fff5a9a9aa None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:28.703262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:28.707924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d43f6-ef5a-4107-bc49-6d849d63b6cf None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:28.774690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e84988-3879-4a9f-ba02-ef33eaf7451c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:28.775332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e84988-3879-4a9f-ba02-ef33eaf7451c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:28.775611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e84988-3879-4a9f-ba02-ef33eaf7451c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:28.775878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e84988-3879-4a9f-ba02-ef33eaf7451c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:28.795484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e84988-3879-4a9f-ba02-ef33eaf7451c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:28.797822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e84988-3879-4a9f-ba02-ef33eaf7451c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-56e84988-3879-4a9f-ba02-ef33eaf7451c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:28.821667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:28.822645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:28.823481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:28.823481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:28.823481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:28.823481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 242/493] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:28.829362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e84988-3879-4a9f-ba02-ef33eaf7451c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:28.864060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18154b05-59b4-4ca0-b81e-d8fff5a9a9aa None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:28.918514 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f40b18-b1ef-42ed-97ae-f4f7ce294d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:28.919654 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f40b18-b1ef-42ed-97ae-f4f7ce294d05 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:28.920374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f40b18-b1ef-42ed-97ae-f4f7ce294d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:28.921092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f40b18-b1ef-42ed-97ae-f4f7ce294d05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:28.938574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f40b18-b1ef-42ed-97ae-f4f7ce294d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:28.940385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f40b18-b1ef-42ed-97ae-f4f7ce294d05 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e0f40b18-b1ef-42ed-97ae-f4f7ce294d05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:28.957498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:28.959165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:28.960266 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:28.960266 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:28.960266 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:28.960266 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 252/494] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:28.960266 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f40b18-b1ef-42ed-97ae-f4f7ce294d05 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:28.969944 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e84988-3879-4a9f-ba02-ef33eaf7451c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:28.996505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:29.010890 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a529409-5770-4d5d-9b8e-1dcb7b1b8a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:29.011508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a529409-5770-4d5d-9b8e-1dcb7b1b8a18 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:29.011804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a529409-5770-4d5d-9b8e-1dcb7b1b8a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:29.012074 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a529409-5770-4d5d-9b8e-1dcb7b1b8a18 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:29.012995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a529409-5770-4d5d-9b8e-1dcb7b1b8a18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:29.013319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a529409-5770-4d5d-9b8e-1dcb7b1b8a18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:29.014891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a529409-5770-4d5d-9b8e-1dcb7b1b8a18 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:29.026040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:29.028916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 243/495] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:29.029642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a529409-5770-4d5d-9b8e-1dcb7b1b8a18 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:29.070230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f40b18-b1ef-42ed-97ae-f4f7ce294d05 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:29.108287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbf025-ffe3-461e-bcd5-93d9b3fdead7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:29.108841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbf025-ffe3-461e-bcd5-93d9b3fdead7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:29.109124 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbf025-ffe3-461e-bcd5-93d9b3fdead7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:29.109553 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbf025-ffe3-461e-bcd5-93d9b3fdead7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:29.121335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbf025-ffe3-461e-bcd5-93d9b3fdead7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-06dbf025-ffe3-461e-bcd5-93d9b3fdead7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:29.126962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:29.127826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:29.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:29.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:29.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:29.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:29.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:29.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:29.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:29.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:29.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:29.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:29.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:29.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:29.128608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 253/496] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:29.130089 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbf025-ffe3-461e-bcd5-93d9b3fdead7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:29.154078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a529409-5770-4d5d-9b8e-1dcb7b1b8a18 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:29.191482 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5589b8-4035-4838-859e-a9a34d2dbaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:29.192222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5589b8-4035-4838-859e-a9a34d2dbaac None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:29.192398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5589b8-4035-4838-859e-a9a34d2dbaac None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:29.192677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5589b8-4035-4838-859e-a9a34d2dbaac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:29.203515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5589b8-4035-4838-859e-a9a34d2dbaac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6e5589b8-4035-4838-859e-a9a34d2dbaac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:29.208285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:29.209177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:29.209955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:29.209955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:29.209955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:29.209955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:29.209955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:29.209955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:29.209955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:29.209955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:29.209955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:29.209955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:29.209955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:29.209955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:29.209955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 244/497] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:29.210593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5589b8-4035-4838-859e-a9a34d2dbaac None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:29.238465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbf025-ffe3-461e-bcd5-93d9b3fdead7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:29.277054 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:29.279249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:29.293140 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c35b36-66fe-4d92-9e21-e7fa67044551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:29.294061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c35b36-66fe-4d92-9e21-e7fa67044551 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:29.294397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c35b36-66fe-4d92-9e21-e7fa67044551 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:29.294701 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c35b36-66fe-4d92-9e21-e7fa67044551 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:29.295353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c35b36-66fe-4d92-9e21-e7fa67044551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:29.295703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c35b36-66fe-4d92-9e21-e7fa67044551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:29.297160 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c35b36-66fe-4d92-9e21-e7fa67044551 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:29.315575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 254/498] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:29.317675 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c35b36-66fe-4d92-9e21-e7fa67044551 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:29.343937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5589b8-4035-4838-859e-a9a34d2dbaac None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:29.382906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc4b0a-7439-4a39-a94e-34dbaec52741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:29.383463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc4b0a-7439-4a39-a94e-34dbaec52741 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:29.383746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc4b0a-7439-4a39-a94e-34dbaec52741 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:29.384006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc4b0a-7439-4a39-a94e-34dbaec52741 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:29.391721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc4b0a-7439-4a39-a94e-34dbaec52741 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:29.393341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc4b0a-7439-4a39-a94e-34dbaec52741 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a6bc4b0a-7439-4a39-a94e-34dbaec52741 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:29.406344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:29.407260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:29.408135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:29.408135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:29.408135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:29.408135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:29.408135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:29.408135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:29.408135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:29.408135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:29.408135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:00:29.408135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:29.408135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 245/499] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Sep 6 21:00:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:29.408763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc4b0a-7439-4a39-a94e-34dbaec52741 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:29.432996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c35b36-66fe-4d92-9e21-e7fa67044551 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:29.472166 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28416ff3-902a-4f5f-ad93-8b4dc65bc08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:29.472822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28416ff3-902a-4f5f-ad93-8b4dc65bc08e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:29.473082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28416ff3-902a-4f5f-ad93-8b4dc65bc08e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:29.473350 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28416ff3-902a-4f5f-ad93-8b4dc65bc08e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:29.474029 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28416ff3-902a-4f5f-ad93-8b4dc65bc08e None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:29.474327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28416ff3-902a-4f5f-ad93-8b4dc65bc08e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:29.475701 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28416ff3-902a-4f5f-ad93-8b4dc65bc08e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:29.483856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 255/500] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:29.484681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28416ff3-902a-4f5f-ad93-8b4dc65bc08e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:29.514746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc4b0a-7439-4a39-a94e-34dbaec52741 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:29.517369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:29.532051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:29.550110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76449bb-b4ac-4f10-a770-4f3cf0dfaa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:29.550726 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76449bb-b4ac-4f10-a770-4f3cf0dfaa24 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:29.551191 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76449bb-b4ac-4f10-a770-4f3cf0dfaa24 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:29.551329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76449bb-b4ac-4f10-a770-4f3cf0dfaa24 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:29.551959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76449bb-b4ac-4f10-a770-4f3cf0dfaa24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:29.552255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76449bb-b4ac-4f10-a770-4f3cf0dfaa24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:29.553690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76449bb-b4ac-4f10-a770-4f3cf0dfaa24 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:29.562099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 246/501] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:29.563297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0cbc2-545b-4995-a678-083766086e12 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:29.563579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0cbc2-545b-4995-a678-083766086e12 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:29.563850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0cbc2-545b-4995-a678-083766086e12 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:29.567049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 247/502] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:29.567049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0cbc2-545b-4995-a678-083766086e12 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:29.593453 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28416ff3-902a-4f5f-ad93-8b4dc65bc08e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:29.627195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffd7e9-7fce-4a6d-9d1a-6c2123b465f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:29.627825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffd7e9-7fce-4a6d-9d1a-6c2123b465f2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:29.628100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffd7e9-7fce-4a6d-9d1a-6c2123b465f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:29.628370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffd7e9-7fce-4a6d-9d1a-6c2123b465f2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:29.636214 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffd7e9-7fce-4a6d-9d1a-6c2123b465f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:29.637707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffd7e9-7fce-4a6d-9d1a-6c2123b465f2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-67ffd7e9-7fce-4a6d-9d1a-6c2123b465f2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:29.650541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:29.651507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:29.652377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:29.652377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:29.652377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:29.652377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:29.652377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:29.652377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:29.652377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:29.652377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:29.652377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 21:00:29.652377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:29.652377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 256/503] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:29.653606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffd7e9-7fce-4a6d-9d1a-6c2123b465f2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:29.677441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0cbc2-545b-4995-a678-083766086e12 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:29.718072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad4c4d-1861-4998-b8f5-2fb113d9e0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:29.718702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad4c4d-1861-4998-b8f5-2fb113d9e0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:29.718977 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad4c4d-1861-4998-b8f5-2fb113d9e0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:29.719250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad4c4d-1861-4998-b8f5-2fb113d9e0c4 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:29.719982 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad4c4d-1861-4998-b8f5-2fb113d9e0c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:29.720430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad4c4d-1861-4998-b8f5-2fb113d9e0c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:29.722534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad4c4d-1861-4998-b8f5-2fb113d9e0c4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:29.759556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffd7e9-7fce-4a6d-9d1a-6c2123b465f2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:29.760115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 248/504] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Sep 6 21:00:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:29.761060 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad4c4d-1861-4998-b8f5-2fb113d9e0c4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:29.767626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:29.791152 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:29.803531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8edb6f-b645-49d0-9d08-d3a356d11bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:29.805500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8edb6f-b645-49d0-9d08-d3a356d11bbb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:29.805500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8edb6f-b645-49d0-9d08-d3a356d11bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:29.805500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8edb6f-b645-49d0-9d08-d3a356d11bbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:29.820479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8edb6f-b645-49d0-9d08-d3a356d11bbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:29.822603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8edb6f-b645-49d0-9d08-d3a356d11bbb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1b8edb6f-b645-49d0-9d08-d3a356d11bbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:29.839337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:29.840518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:29.841423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:29.841423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:29.841423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:29.841423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 257/505] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:29.842058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c011505-9386-4918-b570-10ba572fffce None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:29.842210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c011505-9386-4918-b570-10ba572fffce None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:29.842478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c011505-9386-4918-b570-10ba572fffce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:29.863257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8c011505-9386-4918-b570-10ba572fffce None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:29.864468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8c011505-9386-4918-b570-10ba572fffce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:29.905252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad4c4d-1861-4998-b8f5-2fb113d9e0c4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:29.939799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 258/506] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:29.940683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c011505-9386-4918-b570-10ba572fffce None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:29.945736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df694fbd-9f7a-4ed7-911e-bf39079b905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:29.946387 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df694fbd-9f7a-4ed7-911e-bf39079b905f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:29.946664 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df694fbd-9f7a-4ed7-911e-bf39079b905f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:29.946937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df694fbd-9f7a-4ed7-911e-bf39079b905f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:29.958071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df694fbd-9f7a-4ed7-911e-bf39079b905f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:29.959556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df694fbd-9f7a-4ed7-911e-bf39079b905f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-df694fbd-9f7a-4ed7-911e-bf39079b905f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:29.978761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:29.979751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:29.980598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:29.980598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:29.980598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:29.980598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 249/507] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:29.981816 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df694fbd-9f7a-4ed7-911e-bf39079b905f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:30.035864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:30.051975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c011505-9386-4918-b570-10ba572fffce None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:30.079257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:30.087828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2d433-b6c5-418f-b2f7-2aa04c05f721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:30.088440 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2d433-b6c5-418f-b2f7-2aa04c05f721 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:30.088906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2d433-b6c5-418f-b2f7-2aa04c05f721 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:30.089075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2d433-b6c5-418f-b2f7-2aa04c05f721 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:30.089759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2d433-b6c5-418f-b2f7-2aa04c05f721 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:30.090061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2d433-b6c5-418f-b2f7-2aa04c05f721 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:30.091576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2d433-b6c5-418f-b2f7-2aa04c05f721 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:30.105354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df694fbd-9f7a-4ed7-911e-bf39079b905f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:30.134771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 259/508] 10.222.0.79 () {56 vars in 1137 bytes} [Tue Sep 6 21:00:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:30.135728 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2d433-b6c5-418f-b2f7-2aa04c05f721 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:30.146804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ad73e-de05-4d23-aec6-8e3d833d3cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:30.147425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ad73e-de05-4d23-aec6-8e3d833d3cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:30.147708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ad73e-de05-4d23-aec6-8e3d833d3cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:30.147975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ad73e-de05-4d23-aec6-8e3d833d3cf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:30.239338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ad73e-de05-4d23-aec6-8e3d833d3cf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:30.240797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ad73e-de05-4d23-aec6-8e3d833d3cf1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:30.253006 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2d433-b6c5-418f-b2f7-2aa04c05f721 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:30.290742 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25281c-fee3-4d12-9e51-9abaf33b8ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:30.291620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25281c-fee3-4d12-9e51-9abaf33b8ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:30.291780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25281c-fee3-4d12-9e51-9abaf33b8ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:30.291980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25281c-fee3-4d12-9e51-9abaf33b8ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:30.292698 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25281c-fee3-4d12-9e51-9abaf33b8ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:30.293094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25281c-fee3-4d12-9e51-9abaf33b8ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:30.294575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25281c-fee3-4d12-9e51-9abaf33b8ca8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:30.306586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 260/509] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:30.307661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25281c-fee3-4d12-9e51-9abaf33b8ca8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:30.313569 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:30.324241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:30.348860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 250/510] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:30.349800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ad73e-de05-4d23-aec6-8e3d833d3cf1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:30.431645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25281c-fee3-4d12-9e51-9abaf33b8ca8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:30.465581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ad73e-de05-4d23-aec6-8e3d833d3cf1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:30.466256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857272f-d473-4ac5-9084-758535849273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:30.466854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857272f-d473-4ac5-9084-758535849273 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:30.467138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857272f-d473-4ac5-9084-758535849273 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:30.467421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857272f-d473-4ac5-9084-758535849273 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:30.468062 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857272f-d473-4ac5-9084-758535849273 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:30.468355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857272f-d473-4ac5-9084-758535849273 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:30.469769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857272f-d473-4ac5-9084-758535849273 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:30.482728 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 261/511] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:30.483579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857272f-d473-4ac5-9084-758535849273 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:30.505267 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529627e-5210-493d-b918-f307ea8b2252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:30.505883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529627e-5210-493d-b918-f307ea8b2252 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:30.506157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529627e-5210-493d-b918-f307ea8b2252 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:30.506424 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529627e-5210-493d-b918-f307ea8b2252 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:30.544180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:30.550390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:30.575493 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529627e-5210-493d-b918-f307ea8b2252 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:30.577020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529627e-5210-493d-b918-f307ea8b2252 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:30.590379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857272f-d473-4ac5-9084-758535849273 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:30.624431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1cb25-7c51-42dd-90b3-7e9f3e3cbc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:30.625102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1cb25-7c51-42dd-90b3-7e9f3e3cbc22 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:30.625379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1cb25-7c51-42dd-90b3-7e9f3e3cbc22 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:30.625652 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1cb25-7c51-42dd-90b3-7e9f3e3cbc22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:30.635580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1cb25-7c51-42dd-90b3-7e9f3e3cbc22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:30.637071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1cb25-7c51-42dd-90b3-7e9f3e3cbc22 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-26b1cb25-7c51-42dd-90b3-7e9f3e3cbc22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:30.651576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:30.652466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:30.653561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:30.653561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:30.653561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:30.653561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 262/512] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:30.653561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1cb25-7c51-42dd-90b3-7e9f3e3cbc22 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:30.678592 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 251/513] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:30.679445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529627e-5210-493d-b918-f307ea8b2252 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:30.753758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1cb25-7c51-42dd-90b3-7e9f3e3cbc22 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:30.754795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:30.770412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:30.779951 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529627e-5210-493d-b918-f307ea8b2252 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:30.791228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb9947-2044-41d4-933a-4f31456c5d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:30.791866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb9947-2044-41d4-933a-4f31456c5d26 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:30.792143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb9947-2044-41d4-933a-4f31456c5d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:30.792416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb9947-2044-41d4-933a-4f31456c5d26 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:30.800399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb9947-2044-41d4-933a-4f31456c5d26 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:30.801872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb9947-2044-41d4-933a-4f31456c5d26 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-65bb9947-2044-41d4-933a-4f31456c5d26 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:30.813783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:30.814740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:30.815658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:30.815658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:30.815658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:30.815658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:30.815658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:30.815658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:30.815658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:30.815658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:30.815658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 21:00:30.815658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:30.816183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b194628-80b0-49ea-8287-99df2dbba9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:30.816183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b194628-80b0-49ea-8287-99df2dbba9df None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:30.816183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b194628-80b0-49ea-8287-99df2dbba9df None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:30.816390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 263/514] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:30.816456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b194628-80b0-49ea-8287-99df2dbba9df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:30.816527 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb9947-2044-41d4-933a-4f31456c5d26 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:30.828058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b194628-80b0-49ea-8287-99df2dbba9df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:30.829556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b194628-80b0-49ea-8287-99df2dbba9df None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6b194628-80b0-49ea-8287-99df2dbba9df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:30.844517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:30.845476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:30.846349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:30.846349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:30.846349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:30.846349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 252/515] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:30.846576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b194628-80b0-49ea-8287-99df2dbba9df None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:30.926471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb9947-2044-41d4-933a-4f31456c5d26 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:30.944429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b194628-80b0-49ea-8287-99df2dbba9df None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:30.960313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b4fb2-a9c3-4c47-8578-c03bf3b231fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:30.960972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b4fb2-a9c3-4c47-8578-c03bf3b231fe None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:30.961249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b4fb2-a9c3-4c47-8578-c03bf3b231fe None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:30.961502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b4fb2-a9c3-4c47-8578-c03bf3b231fe None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:30.969813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b4fb2-a9c3-4c47-8578-c03bf3b231fe None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:30.970448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:30.971258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b4fb2-a9c3-4c47-8578-c03bf3b231fe None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:30.979815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd28fef-131b-4e2a-ac66-5afe84dd694c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:30.980432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd28fef-131b-4e2a-ac66-5afe84dd694c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:30.980746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd28fef-131b-4e2a-ac66-5afe84dd694c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:30.981021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd28fef-131b-4e2a-ac66-5afe84dd694c None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:30.981697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd28fef-131b-4e2a-ac66-5afe84dd694c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:30.982000 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd28fef-131b-4e2a-ac66-5afe84dd694c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-013b4fb2-a9c3-4c47-8578-c03bf3b231fe None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:30.982889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:30.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:30.984629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:30.984629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:30.984629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:30.984629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:30.984629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:30.984629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:30.984629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:30.984629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:30.984629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 21:00:30.984629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:30.985848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd28fef-131b-4e2a-ac66-5afe84dd694c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:30.985927 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 264/516] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:30.985927 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b4fb2-a9c3-4c47-8578-c03bf3b231fe None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:30.996299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 253/517] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:30.997079 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd28fef-131b-4e2a-ac66-5afe84dd694c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:31.021814 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:31.090510 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b4fb2-a9c3-4c47-8578-c03bf3b231fe None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:31.095513 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd28fef-131b-4e2a-ac66-5afe84dd694c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:31.126003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c5d98-9c74-4b9d-b29e-8f3cca026327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:31.126642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c5d98-9c74-4b9d-b29e-8f3cca026327 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:31.126923 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c5d98-9c74-4b9d-b29e-8f3cca026327 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:31.127195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c5d98-9c74-4b9d-b29e-8f3cca026327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:31.130340 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e63c2-2147-442f-a2ab-ed0f9e25cccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:31.130943 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e63c2-2147-442f-a2ab-ed0f9e25cccf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:31.131216 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e63c2-2147-442f-a2ab-ed0f9e25cccf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:31.131482 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e63c2-2147-442f-a2ab-ed0f9e25cccf None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:31.132200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e63c2-2147-442f-a2ab-ed0f9e25cccf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:31.132499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e63c2-2147-442f-a2ab-ed0f9e25cccf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:31.134166 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e63c2-2147-442f-a2ab-ed0f9e25cccf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:31.137496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c5d98-9c74-4b9d-b29e-8f3cca026327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:31.138990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c5d98-9c74-4b9d-b29e-8f3cca026327 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:31.151809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 254/518] 10.222.0.79 () {56 vars in 1137 bytes} [Tue Sep 6 21:00:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:31.152600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e63c2-2147-442f-a2ab-ed0f9e25cccf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-222c5d98-9c74-4b9d-b29e-8f3cca026327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:31.153768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:31.154661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:31.155498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:31.155498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:31.155498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:31.155498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 265/519] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:31.156160 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c5d98-9c74-4b9d-b29e-8f3cca026327 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:31.202152 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:31.247128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:31.254138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e63c2-2147-442f-a2ab-ed0f9e25cccf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:31.259246 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c5d98-9c74-4b9d-b29e-8f3cca026327 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:31.290530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff500f57-4b2a-4cf9-9e4d-5f58691ba615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:31.291066 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff500f57-4b2a-4cf9-9e4d-5f58691ba615 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:31.291407 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff500f57-4b2a-4cf9-9e4d-5f58691ba615 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:31.291727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff500f57-4b2a-4cf9-9e4d-5f58691ba615 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:31.292371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff500f57-4b2a-4cf9-9e4d-5f58691ba615 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:31.292691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff500f57-4b2a-4cf9-9e4d-5f58691ba615 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:31.293500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c45ff-fe0c-4a89-a533-f309ad8fdbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:31.294214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff500f57-4b2a-4cf9-9e4d-5f58691ba615 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:31.294306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c45ff-fe0c-4a89-a533-f309ad8fdbaa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:31.294489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c45ff-fe0c-4a89-a533-f309ad8fdbaa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:31.294760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c45ff-fe0c-4a89-a533-f309ad8fdbaa None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:31.295490 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c45ff-fe0c-4a89-a533-f309ad8fdbaa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:31.295789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c45ff-fe0c-4a89-a533-f309ad8fdbaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:31.297243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c45ff-fe0c-4a89-a533-f309ad8fdbaa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:31.305441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 255/520] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:31.306163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff500f57-4b2a-4cf9-9e4d-5f58691ba615 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:31.319315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 266/521] 10.222.0.79 () {56 vars in 1139 bytes} [Tue Sep 6 21:00:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:31.320153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c45ff-fe0c-4a89-a533-f309ad8fdbaa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:31.406398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff500f57-4b2a-4cf9-9e4d-5f58691ba615 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:31.413214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:31.422227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c45ff-fe0c-4a89-a533-f309ad8fdbaa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:31.440091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0fabb-501f-4eea-bc7d-1e17d0ad2a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:31.440720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0fabb-501f-4eea-bc7d-1e17d0ad2a20 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:31.440998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0fabb-501f-4eea-bc7d-1e17d0ad2a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:31.441264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0fabb-501f-4eea-bc7d-1e17d0ad2a20 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:31.441937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0fabb-501f-4eea-bc7d-1e17d0ad2a20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:31.443371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0fabb-501f-4eea-bc7d-1e17d0ad2a20 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:31.457717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c5256-9f2b-469d-a96d-74de135efb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:31.458328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c5256-9f2b-469d-a96d-74de135efb98 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:31.458647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c5256-9f2b-469d-a96d-74de135efb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:31.458933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c5256-9f2b-469d-a96d-74de135efb98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:31.476697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 256/522] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 21:00:31 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:00:31.476697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0fabb-501f-4eea-bc7d-1e17d0ad2a20 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:31.476967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:31.538937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c5256-9f2b-469d-a96d-74de135efb98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:31.540433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c5256-9f2b-469d-a96d-74de135efb98 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:31.584131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0fabb-501f-4eea-bc7d-1e17d0ad2a20 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:31.618864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6cc87-4e4b-4ae8-9cd9-45577eaffbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:31.619596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6cc87-4e4b-4ae8-9cd9-45577eaffbaf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:31.619949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6cc87-4e4b-4ae8-9cd9-45577eaffbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:31.620270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6cc87-4e4b-4ae8-9cd9-45577eaffbaf None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:31.621134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6cc87-4e4b-4ae8-9cd9-45577eaffbaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:31.621479 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6cc87-4e4b-4ae8-9cd9-45577eaffbaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:31.623026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6cc87-4e4b-4ae8-9cd9-45577eaffbaf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:31.636168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 257/523] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:31.637064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6cc87-4e4b-4ae8-9cd9-45577eaffbaf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:31.663191 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:31.672019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 267/524] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:31.672807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c5256-9f2b-469d-a96d-74de135efb98 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:31.727008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:31.750488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6cc87-4e4b-4ae8-9cd9-45577eaffbaf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:31.790265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c5256-9f2b-469d-a96d-74de135efb98 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:31.793891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d24ded-082d-44de-8290-036898da4bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:31.794844 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d24ded-082d-44de-8290-036898da4bfc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:31.795324 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d24ded-082d-44de-8290-036898da4bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:31.795792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d24ded-082d-44de-8290-036898da4bfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:31.805125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d24ded-082d-44de-8290-036898da4bfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:31.807518 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d24ded-082d-44de-8290-036898da4bfc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d0d24ded-082d-44de-8290-036898da4bfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:31.821967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:31.822837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:31.823752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:31.823752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:31.823752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:31.823752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:31.823752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:31.823752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:31.823752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:31.823752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:31.823752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 258/525] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:31.825094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d24ded-082d-44de-8290-036898da4bfc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:31.830805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e7a1a-b93a-42ca-bca1-02891b430201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:31.831422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e7a1a-b93a-42ca-bca1-02891b430201 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:31.831730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e7a1a-b93a-42ca-bca1-02891b430201 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:31.831972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e7a1a-b93a-42ca-bca1-02891b430201 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:31.912205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e7a1a-b93a-42ca-bca1-02891b430201 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:31.913623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e7a1a-b93a-42ca-bca1-02891b430201 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:31.944740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:31.964713 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d24ded-082d-44de-8290-036898da4bfc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:31.976615 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:32.001338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197158e-c2fe-4166-8784-42a0ee5f352b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:32.002278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197158e-c2fe-4166-8784-42a0ee5f352b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:32.002657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197158e-c2fe-4166-8784-42a0ee5f352b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:32.002866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197158e-c2fe-4166-8784-42a0ee5f352b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:32.009881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197158e-c2fe-4166-8784-42a0ee5f352b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:32.011390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197158e-c2fe-4166-8784-42a0ee5f352b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5197158e-c2fe-4166-8784-42a0ee5f352b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:32.024562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:32.025629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:32.026563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:32.026563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:32.026563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:32.026563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:32.026563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:32.026563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:32.026563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:32.026563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:32.026563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 259/526] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:32.026563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197158e-c2fe-4166-8784-42a0ee5f352b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:32.059390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 268/527] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:32.059390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e7a1a-b93a-42ca-bca1-02891b430201 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:32.159186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197158e-c2fe-4166-8784-42a0ee5f352b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:32.165145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e7a1a-b93a-42ca-bca1-02891b430201 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:32.194342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01e139-551d-41f9-9206-18498cc3cc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:32.194977 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01e139-551d-41f9-9206-18498cc3cc5e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:32.195255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01e139-551d-41f9-9206-18498cc3cc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:32.195531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01e139-551d-41f9-9206-18498cc3cc5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:32.200188 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437674b0-2b7e-4223-abf5-f01d50732996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:32.200841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437674b0-2b7e-4223-abf5-f01d50732996 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:32.201119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437674b0-2b7e-4223-abf5-f01d50732996 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:32.201386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437674b0-2b7e-4223-abf5-f01d50732996 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:32.202073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437674b0-2b7e-4223-abf5-f01d50732996 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:32.202372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437674b0-2b7e-4223-abf5-f01d50732996 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:32.203819 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437674b0-2b7e-4223-abf5-f01d50732996 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:32.206971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01e139-551d-41f9-9206-18498cc3cc5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:32.208444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01e139-551d-41f9-9206-18498cc3cc5e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:32.210544 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:32.222508 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 269/528] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:32.223368 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437674b0-2b7e-4223-abf5-f01d50732996 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2f01e139-551d-41f9-9206-18498cc3cc5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:32.225488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:32.226390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:32.227248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:32.227248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:32.227248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:32.229061 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 260/529] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:32.230085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01e139-551d-41f9-9206-18498cc3cc5e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:32.243096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:32.345056 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437674b0-2b7e-4223-abf5-f01d50732996 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:32.345926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01e139-551d-41f9-9206-18498cc3cc5e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:32.386981 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c20bca-c7f9-4b19-8b9f-e408f1e4adf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:32.387640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c20bca-c7f9-4b19-8b9f-e408f1e4adf4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:32.387911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c20bca-c7f9-4b19-8b9f-e408f1e4adf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:32.388184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c20bca-c7f9-4b19-8b9f-e408f1e4adf4 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:32.388920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c20bca-c7f9-4b19-8b9f-e408f1e4adf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:32.389227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c20bca-c7f9-4b19-8b9f-e408f1e4adf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:32.390810 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c20bca-c7f9-4b19-8b9f-e408f1e4adf4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:32.391931 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa935df-a9e0-4f05-8a4f-1b47e3aa43a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:32.393034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa935df-a9e0-4f05-8a4f-1b47e3aa43a8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:32.393596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa935df-a9e0-4f05-8a4f-1b47e3aa43a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:32.394005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa935df-a9e0-4f05-8a4f-1b47e3aa43a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:32.405579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 270/530] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:32.406461 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c20bca-c7f9-4b19-8b9f-e408f1e4adf4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:32.409989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa935df-a9e0-4f05-8a4f-1b47e3aa43a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:32.413019 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa935df-a9e0-4f05-8a4f-1b47e3aa43a8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-caa935df-a9e0-4f05-8a4f-1b47e3aa43a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:32.432305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:32.433794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:32.434697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:32.434697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:32.434697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:32.434697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 261/531] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:32.434918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa935df-a9e0-4f05-8a4f-1b47e3aa43a8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:32.487691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:32.490514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:32.524380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c20bca-c7f9-4b19-8b9f-e408f1e4adf4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:32.554809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa935df-a9e0-4f05-8a4f-1b47e3aa43a8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:32.559446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6920115-2db4-437c-90a9-1e8aa199468e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:32.560062 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6920115-2db4-437c-90a9-1e8aa199468e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:32.560726 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6920115-2db4-437c-90a9-1e8aa199468e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:32.560996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6920115-2db4-437c-90a9-1e8aa199468e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:32.561741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6920115-2db4-437c-90a9-1e8aa199468e None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=12d51454d20c46d7a5e6d963a2f987cc) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:32.562046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6920115-2db4-437c-90a9-1e8aa199468e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:32.563494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6920115-2db4-437c-90a9-1e8aa199468e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:32.569214 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 271/532] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:32 2022] GET /identity/v3/role_assignments?user.id=12d51454d20c46d7a5e6d963a2f987cc&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:32.569823 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6920115-2db4-437c-90a9-1e8aa199468e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:32.612565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f072fd0-c3a7-451e-b1ff-f28017ba7298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:32.613644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f072fd0-c3a7-451e-b1ff-f28017ba7298 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:32.614124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f072fd0-c3a7-451e-b1ff-f28017ba7298 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:32.614594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f072fd0-c3a7-451e-b1ff-f28017ba7298 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:32.615477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f072fd0-c3a7-451e-b1ff-f28017ba7298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:32.615976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f072fd0-c3a7-451e-b1ff-f28017ba7298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:32.617655 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f072fd0-c3a7-451e-b1ff-f28017ba7298 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:32.631529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 262/533] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:32.632903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f072fd0-c3a7-451e-b1ff-f28017ba7298 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:32.706559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6920115-2db4-437c-90a9-1e8aa199468e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:32.740464 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:32.743548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f072fd0-c3a7-451e-b1ff-f28017ba7298 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:32.749291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410e682-3036-4aa6-8184-1719270a4ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:32.750116 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410e682-3036-4aa6-8184-1719270a4ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:32.750562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410e682-3036-4aa6-8184-1719270a4ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:32.750971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410e682-3036-4aa6-8184-1719270a4ee8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:32.751849 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410e682-3036-4aa6-8184-1719270a4ee8 None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=a76937ab52e445c59b9eb9638fd4a753) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:32.752303 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410e682-3036-4aa6-8184-1719270a4ee8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:32.753564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:32.754803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410e682-3036-4aa6-8184-1719270a4ee8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:32.762519 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 272/534] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:32 2022] GET /identity/v3/role_assignments?user.id=a76937ab52e445c59b9eb9638fd4a753&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:32.763571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410e682-3036-4aa6-8184-1719270a4ee8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:32.789796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b214eb-3dd9-4297-b5e7-10ae73fa9054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:32.789796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b214eb-3dd9-4297-b5e7-10ae73fa9054 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:32.789796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b214eb-3dd9-4297-b5e7-10ae73fa9054 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:32.790492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b214eb-3dd9-4297-b5e7-10ae73fa9054 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:32.791314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b214eb-3dd9-4297-b5e7-10ae73fa9054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:32.791764 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b214eb-3dd9-4297-b5e7-10ae73fa9054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:32.793463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b214eb-3dd9-4297-b5e7-10ae73fa9054 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:32.806463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 263/535] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:32.813178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b214eb-3dd9-4297-b5e7-10ae73fa9054 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:32.895261 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410e682-3036-4aa6-8184-1719270a4ee8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:32.936133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956837b-411e-4f48-bcda-95e360537f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:32.937197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956837b-411e-4f48-bcda-95e360537f64 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:32.937700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956837b-411e-4f48-bcda-95e360537f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:32.938163 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956837b-411e-4f48-bcda-95e360537f64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:32.946852 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956837b-411e-4f48-bcda-95e360537f64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:32.949720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956837b-411e-4f48-bcda-95e360537f64 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1956837b-411e-4f48-bcda-95e360537f64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:32.965106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:32.966118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:32.966948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:32.966948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:32.966948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:32.966948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:32.966948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:32.966948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:32.966948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:32.966948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:32.969578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 273/536] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:32.975736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956837b-411e-4f48-bcda-95e360537f64 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:33.030734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b214eb-3dd9-4297-b5e7-10ae73fa9054 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:33.080776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:33.083951 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a3903-2385-4d46-9be6-bebc7167c046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:33.084608 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a3903-2385-4d46-9be6-bebc7167c046 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:33.084906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a3903-2385-4d46-9be6-bebc7167c046 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:33.085175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a3903-2385-4d46-9be6-bebc7167c046 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:33.101887 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a3903-2385-4d46-9be6-bebc7167c046 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:33.103371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a3903-2385-4d46-9be6-bebc7167c046 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3f8a3903-2385-4d46-9be6-bebc7167c046 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:33.133663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:33.134565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:33.135373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:33.135373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:33.135373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:33.135373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:33.135373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:33.135373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:33.135373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:33.135373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:33.139494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 264/537] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:00:33.142828 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a3903-2385-4d46-9be6-bebc7167c046 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:33.184272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956837b-411e-4f48-bcda-95e360537f64 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:33.205153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:33.241612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf5f10-f07c-42bc-b667-9abda64d2966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:33.242181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf5f10-f07c-42bc-b667-9abda64d2966 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:33.242496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf5f10-f07c-42bc-b667-9abda64d2966 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:33.242866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf5f10-f07c-42bc-b667-9abda64d2966 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:33.243672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf5f10-f07c-42bc-b667-9abda64d2966 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:33.244008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf5f10-f07c-42bc-b667-9abda64d2966 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:33.248688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf5f10-f07c-42bc-b667-9abda64d2966 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:33.261304 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 274/538] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:33.267469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf5f10-f07c-42bc-b667-9abda64d2966 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:33.351502 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a3903-2385-4d46-9be6-bebc7167c046 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:33.394943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf5f10-f07c-42bc-b667-9abda64d2966 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:33.415364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bdf0b-3663-4680-b9d8-b8abecfe4b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:33.416004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bdf0b-3663-4680-b9d8-b8abecfe4b78 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:33.416719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bdf0b-3663-4680-b9d8-b8abecfe4b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:33.417000 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bdf0b-3663-4680-b9d8-b8abecfe4b78 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:33.417725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bdf0b-3663-4680-b9d8-b8abecfe4b78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:33.418024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bdf0b-3663-4680-b9d8-b8abecfe4b78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:33.419548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bdf0b-3663-4680-b9d8-b8abecfe4b78 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:33.425621 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:33.444826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 265/539] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:33.450910 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bdf0b-3663-4680-b9d8-b8abecfe4b78 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:33.476038 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485c520-9ad6-46aa-ad07-02052d97140b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:33.476883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485c520-9ad6-46aa-ad07-02052d97140b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:33.477274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485c520-9ad6-46aa-ad07-02052d97140b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:33.479051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485c520-9ad6-46aa-ad07-02052d97140b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:33.479897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485c520-9ad6-46aa-ad07-02052d97140b None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=f50ae3b9997d4dc39c3d67ca40bbc566) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:33.480332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485c520-9ad6-46aa-ad07-02052d97140b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:33.482585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485c520-9ad6-46aa-ad07-02052d97140b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:33.494047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 275/540] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:33 2022] GET /identity/v3/role_assignments?user.id=f50ae3b9997d4dc39c3d67ca40bbc566&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:33.581381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:33.605552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bdf0b-3663-4680-b9d8-b8abecfe4b78 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:33.649226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6acb7-a999-433f-bba2-4b60e54915d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:33.650057 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6acb7-a999-433f-bba2-4b60e54915d9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:33.650431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6acb7-a999-433f-bba2-4b60e54915d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:33.650801 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6acb7-a999-433f-bba2-4b60e54915d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:33.651680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6acb7-a999-433f-bba2-4b60e54915d9 None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=d5a45dec40844877a6b5b301fc44a110) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:33.652089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6acb7-a999-433f-bba2-4b60e54915d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:33.653822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6acb7-a999-433f-bba2-4b60e54915d9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:33.660941 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 266/541] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:33 2022] GET /identity/v3/role_assignments?user.id=d5a45dec40844877a6b5b301fc44a110&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:33.914537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bcde9-6218-44bb-8772-bce840453173 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:33.915078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bcde9-6218-44bb-8772-bce840453173 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:33.915371 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bcde9-6218-44bb-8772-bce840453173 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:33.917157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 276/542] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:33.923210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fea35-9ddb-4a83-903c-0191a9564cba None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:33.923721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fea35-9ddb-4a83-903c-0191a9564cba None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:33.923721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fea35-9ddb-4a83-903c-0191a9564cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:33.950958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-354fea35-9ddb-4a83-903c-0191a9564cba None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:33.952226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-354fea35-9ddb-4a83-903c-0191a9564cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:34.031134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 267/543] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 21:00:34.360181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6bcde9-6218-44bb-8772-bce840453173 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:34.489353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6bcde9-6218-44bb-8772-bce840453173 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:34.535303 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49680b2-ddc6-4957-839b-851ebc93ab48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:34.535932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49680b2-ddc6-4957-839b-851ebc93ab48 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:34.536205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49680b2-ddc6-4957-839b-851ebc93ab48 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:34.536468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49680b2-ddc6-4957-839b-851ebc93ab48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:34.555219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49680b2-ddc6-4957-839b-851ebc93ab48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b49680b2-ddc6-4957-839b-851ebc93ab48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:34.560136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:34.561436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:34.562230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:34.562230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:34.562230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:34.562230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:34.562230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:34.562230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:34.562230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:34.562230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:34.562230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:34.562230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:34.562230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:34.562230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:34.562230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 277/544] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:34.569280 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fea35-9ddb-4a83-903c-0191a9564cba None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:34.689037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:34.713656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fea35-9ddb-4a83-903c-0191a9564cba None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:34.762877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6701c-44a7-43a1-b218-af94d458b5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:34.763503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6701c-44a7-43a1-b218-af94d458b5ff None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:34.763780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6701c-44a7-43a1-b218-af94d458b5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:34.764049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6701c-44a7-43a1-b218-af94d458b5ff None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:34.764744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6701c-44a7-43a1-b218-af94d458b5ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:34.765045 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6701c-44a7-43a1-b218-af94d458b5ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:34.766503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6701c-44a7-43a1-b218-af94d458b5ff None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:34.775946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 268/545] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:34.782076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49680b2-ddc6-4957-839b-851ebc93ab48 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:34.886972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:34.972527 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49680b2-ddc6-4957-839b-851ebc93ab48 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:35.016389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b682cda-f11e-43db-b07b-01a357620310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:35.017055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b682cda-f11e-43db-b07b-01a357620310 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:35.017328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b682cda-f11e-43db-b07b-01a357620310 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:35.017592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b682cda-f11e-43db-b07b-01a357620310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:35.030659 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b682cda-f11e-43db-b07b-01a357620310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:35.032113 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b682cda-f11e-43db-b07b-01a357620310 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6b682cda-f11e-43db-b07b-01a357620310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:35.052759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:35.053704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:35.054515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:35.054515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:35.054515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:35.054515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 278/546] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:35.059636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6701c-44a7-43a1-b218-af94d458b5ff None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:35.210351 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6701c-44a7-43a1-b218-af94d458b5ff None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:35.259245 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76225e-b3db-42cf-aec9-187def0dcd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:35.260033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76225e-b3db-42cf-aec9-187def0dcd6f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:35.260310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76225e-b3db-42cf-aec9-187def0dcd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:35.260576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76225e-b3db-42cf-aec9-187def0dcd6f None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:35.261241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76225e-b3db-42cf-aec9-187def0dcd6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:35.261538 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76225e-b3db-42cf-aec9-187def0dcd6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:35.262959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76225e-b3db-42cf-aec9-187def0dcd6f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:35.275920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 269/547] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:35.285784 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b682cda-f11e-43db-b07b-01a357620310 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:35.314872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2ad21-2bc0-44ff-83e9-d8d3bb742122 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:35.315189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2ad21-2bc0-44ff-83e9-d8d3bb742122 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:35.315419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2ad21-2bc0-44ff-83e9-d8d3bb742122 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:35.316916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 270/548] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:35.334180 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce0902-4caf-41f9-95f4-e84675029b53 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:35.334463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce0902-4caf-41f9-95f4-e84675029b53 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:35.334739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce0902-4caf-41f9-95f4-e84675029b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:35.350389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:35.369645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-65ce0902-4caf-41f9-95f4-e84675029b53 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:35.370925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-65ce0902-4caf-41f9-95f4-e84675029b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:35.389528 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:35.399772 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b682cda-f11e-43db-b07b-01a357620310 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:35.449522 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685d032-f955-4cae-b926-2064df397dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:35.450200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685d032-f955-4cae-b926-2064df397dda None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:35.450477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685d032-f955-4cae-b926-2064df397dda None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:35.450746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685d032-f955-4cae-b926-2064df397dda None admin] PATH_INFO: `/v3/users/swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:35.459318 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685d032-f955-4cae-b926-2064df397dda None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:35.460947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685d032-f955-4cae-b926-2064df397dda None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9685d032-f955-4cae-b926-2064df397dda None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:35.475212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:35.476430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:35.477366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:35.477366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:35.477366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:35.477366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:35.477366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:35.477366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:35.477366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:35.477366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:35.477366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 21:00:35.477366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:35.477366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 279/549] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:00:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:35.482786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685d032-f955-4cae-b926-2064df397dda None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:35.510396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 271/550] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 21:00:35.596785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685d032-f955-4cae-b926-2064df397dda None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:35.631283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eaa801-4ef0-44fd-a906-5db534b2afeb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:35.631474 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d57815-dbfd-4ec2-a5c4-baccac4ca417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:35.632051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eaa801-4ef0-44fd-a906-5db534b2afeb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:35.632154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d57815-dbfd-4ec2-a5c4-baccac4ca417 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:35.632347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d57815-dbfd-4ec2-a5c4-baccac4ca417 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:35.632603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d57815-dbfd-4ec2-a5c4-baccac4ca417 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:35.633018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eaa801-4ef0-44fd-a906-5db534b2afeb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:35.633387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d57815-dbfd-4ec2-a5c4-baccac4ca417 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:35.633690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d57815-dbfd-4ec2-a5c4-baccac4ca417 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:35.635206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 272/551] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:35.639488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d57815-dbfd-4ec2-a5c4-baccac4ca417 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:35.640995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2edb29-02c6-4015-bdcc-c0d76ec4e6fc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:35.641506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2edb29-02c6-4015-bdcc-c0d76ec4e6fc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:35.641977 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2edb29-02c6-4015-bdcc-c0d76ec4e6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:35.658220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 280/552] 10.222.0.79 () {56 vars in 1135 bytes} [Tue Sep 6 21:00:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:35.663157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d57815-dbfd-4ec2-a5c4-baccac4ca417 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:35.665201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-da2edb29-02c6-4015-bdcc-c0d76ec4e6fc None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:35.666620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-da2edb29-02c6-4015-bdcc-c0d76ec4e6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:35.682368 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:35.784226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 273/553] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 21:00:35.843126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d57815-dbfd-4ec2-a5c4-baccac4ca417 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:35.849988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2edb29-02c6-4015-bdcc-c0d76ec4e6fc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:35.914859 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fc548-26e7-438d-81df-61c5838b1226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:35.915477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fc548-26e7-438d-81df-61c5838b1226 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:35.915755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fc548-26e7-438d-81df-61c5838b1226 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:35.916027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fc548-26e7-438d-81df-61c5838b1226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:35.931943 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:35.998640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2edb29-02c6-4015-bdcc-c0d76ec4e6fc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:36.019915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fc548-26e7-438d-81df-61c5838b1226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:36.022126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fc548-26e7-438d-81df-61c5838b1226 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:36.041420 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:36.066564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d6b3f-8e81-46a9-ba07-98b3ae063dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:36.067976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d6b3f-8e81-46a9-ba07-98b3ae063dcb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:36.068546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d6b3f-8e81-46a9-ba07-98b3ae063dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:36.069064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d6b3f-8e81-46a9-ba07-98b3ae063dcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:36.097844 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d6b3f-8e81-46a9-ba07-98b3ae063dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-170d6b3f-8e81-46a9-ba07-98b3ae063dcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:36.103437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:36.104449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:36.105598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:36.105598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:36.105598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:36.105598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:36.105598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:36.105598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:36.105598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:36.105598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:36.105598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:36.105598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:36.105598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:36.105598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:36.105598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 274/554] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:00:36.111009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d6b3f-8e81-46a9-ba07-98b3ae063dcb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:36.182872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 281/555] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 21:00:36.188957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fc548-26e7-438d-81df-61c5838b1226 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:36.257084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d6b3f-8e81-46a9-ba07-98b3ae063dcb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:36.280833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:36.293203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95eb641-22fc-417f-a61f-97b5c8f099d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:36.293803 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95eb641-22fc-417f-a61f-97b5c8f099d7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:36.294085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95eb641-22fc-417f-a61f-97b5c8f099d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:36.294358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95eb641-22fc-417f-a61f-97b5c8f099d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:36.295031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95eb641-22fc-417f-a61f-97b5c8f099d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:36.295336 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95eb641-22fc-417f-a61f-97b5c8f099d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:36.296775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95eb641-22fc-417f-a61f-97b5c8f099d7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:36.305542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fc548-26e7-438d-81df-61c5838b1226 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:36.306155 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 275/556] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:36.309610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95eb641-22fc-417f-a61f-97b5c8f099d7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:36.317204 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:36.344517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7239677-a173-4810-91ed-93a7d0d3c977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:36.345294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7239677-a173-4810-91ed-93a7d0d3c977 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:36.345575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7239677-a173-4810-91ed-93a7d0d3c977 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:36.345851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7239677-a173-4810-91ed-93a7d0d3c977 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:36.358152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7239677-a173-4810-91ed-93a7d0d3c977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c7239677-a173-4810-91ed-93a7d0d3c977 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:36.363352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:36.364250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:36.365097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:36.365097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:36.365097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:36.365097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:36.365097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:36.365097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:36.365097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:36.365097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:36.365097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:36.365097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:36.365097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:36.365097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:36.365097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 282/557] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:00:36.365779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c2ed-9b2d-44b2-a411-a1e106c75248 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:36.366345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c2ed-9b2d-44b2-a411-a1e106c75248 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:36.366345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c2ed-9b2d-44b2-a411-a1e106c75248 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:36.371587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 283/558] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:36.371587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9c2ed-9b2d-44b2-a411-a1e106c75248 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:36.421634 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95eb641-22fc-417f-a61f-97b5c8f099d7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:36.462450 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac154f-7162-4104-a9af-66c054301e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:36.463193 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac154f-7162-4104-a9af-66c054301e3d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:36.463503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac154f-7162-4104-a9af-66c054301e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:36.463777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac154f-7162-4104-a9af-66c054301e3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:36.476495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac154f-7162-4104-a9af-66c054301e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:36.478331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac154f-7162-4104-a9af-66c054301e3d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:36.489065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9c2ed-9b2d-44b2-a411-a1e106c75248 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e1ac154f-7162-4104-a9af-66c054301e3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:36.493571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:36.494663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:36.495534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:36.495534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:36.495534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:36.495534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 276/559] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:36.495534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac154f-7162-4104-a9af-66c054301e3d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:36.530586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fbf91-4aef-4aa6-b85f-a95e02d16236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:36.531231 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fbf91-4aef-4aa6-b85f-a95e02d16236 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:36.531500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fbf91-4aef-4aa6-b85f-a95e02d16236 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:36.531772 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fbf91-4aef-4aa6-b85f-a95e02d16236 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:36.538743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:36.542703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fbf91-4aef-4aa6-b85f-a95e02d16236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:36.544415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fbf91-4aef-4aa6-b85f-a95e02d16236 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-348fbf91-4aef-4aa6-b85f-a95e02d16236 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:36.561817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:36.562897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:36.563741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:36.563741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:36.563741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:36.563741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 284/560] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:36.564432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92461ea0-7ce1-4cfd-84f7-91f5e8d8e66c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:36.564432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92461ea0-7ce1-4cfd-84f7-91f5e8d8e66c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:36.564683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92461ea0-7ce1-4cfd-84f7-91f5e8d8e66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:36.584242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-92461ea0-7ce1-4cfd-84f7-91f5e8d8e66c None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:36.584458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-92461ea0-7ce1-4cfd-84f7-91f5e8d8e66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:36.598914 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:36.611109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac154f-7162-4104-a9af-66c054301e3d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:36.645135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924535e-c3fe-4cf6-9970-441928ba9828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:36.645754 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924535e-c3fe-4cf6-9970-441928ba9828 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:36.646035 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924535e-c3fe-4cf6-9970-441928ba9828 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:36.646303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924535e-c3fe-4cf6-9970-441928ba9828 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:36.646972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924535e-c3fe-4cf6-9970-441928ba9828 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:36.647275 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924535e-c3fe-4cf6-9970-441928ba9828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:36.648678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924535e-c3fe-4cf6-9970-441928ba9828 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:36.651006 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 285/561] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:36.664744 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92461ea0-7ce1-4cfd-84f7-91f5e8d8e66c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:36.664882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 277/562] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:36.664882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1c884-f293-4cf2-8e61-330deb0952b9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:36.664882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1c884-f293-4cf2-8e61-330deb0952b9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:36.664882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1c884-f293-4cf2-8e61-330deb0952b9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:36.664882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 278/563] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:36.665562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1c884-f293-4cf2-8e61-330deb0952b9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:36.766021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92461ea0-7ce1-4cfd-84f7-91f5e8d8e66c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:36.771418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:36.777100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1c884-f293-4cf2-8e61-330deb0952b9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:36.804796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb2878-6eeb-40a3-a8ad-4af377d390c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:36.805329 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb2878-6eeb-40a3-a8ad-4af377d390c9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:36.805608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb2878-6eeb-40a3-a8ad-4af377d390c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:36.805880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb2878-6eeb-40a3-a8ad-4af377d390c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:36.806528 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb2878-6eeb-40a3-a8ad-4af377d390c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:36.806836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb2878-6eeb-40a3-a8ad-4af377d390c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:36.808263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb2878-6eeb-40a3-a8ad-4af377d390c9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:36.819501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e87db-3ede-4dc0-8a22-79c5501a28bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:36.820450 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e87db-3ede-4dc0-8a22-79c5501a28bb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:36.820583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 286/564] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:36.821070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e87db-3ede-4dc0-8a22-79c5501a28bb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:36.821342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb2878-6eeb-40a3-a8ad-4af377d390c9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:36.821742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e87db-3ede-4dc0-8a22-79c5501a28bb None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:36.822464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e87db-3ede-4dc0-8a22-79c5501a28bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:36.822917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e87db-3ede-4dc0-8a22-79c5501a28bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:36.824398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e87db-3ede-4dc0-8a22-79c5501a28bb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:36.844381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 279/565] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:36.846824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbe472-1952-458c-83a8-7db1abe32d72 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:36.847277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbe472-1952-458c-83a8-7db1abe32d72 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:36.847739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbe472-1952-458c-83a8-7db1abe32d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:36.848557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:36.868428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-aafbe472-1952-458c-83a8-7db1abe32d72 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:36.869658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-aafbe472-1952-458c-83a8-7db1abe32d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:36.948527 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 280/566] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:36.950454 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbe472-1952-458c-83a8-7db1abe32d72 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:36.971467 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb2878-6eeb-40a3-a8ad-4af377d390c9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:37.016633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab18e1-d464-4131-81fd-5a8080589c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:37.017396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab18e1-d464-4131-81fd-5a8080589c66 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:37.017878 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab18e1-d464-4131-81fd-5a8080589c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:37.018209 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab18e1-d464-4131-81fd-5a8080589c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:37.031233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab18e1-d464-4131-81fd-5a8080589c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:37.032789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab18e1-d464-4131-81fd-5a8080589c66 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-03ab18e1-d464-4131-81fd-5a8080589c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:37.052677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:37.053862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:37.054723 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:37.054723 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:37.054723 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:37.054723 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 287/567] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:37.054723 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab18e1-d464-4131-81fd-5a8080589c66 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:37.070684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:37.085857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbe472-1952-458c-83a8-7db1abe32d72 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:37.124094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:37.131951 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567a20d-8c59-432d-b6ee-4843c42e830d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:37.132624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567a20d-8c59-432d-b6ee-4843c42e830d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:37.132906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567a20d-8c59-432d-b6ee-4843c42e830d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:37.133171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567a20d-8c59-432d-b6ee-4843c42e830d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:37.140834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567a20d-8c59-432d-b6ee-4843c42e830d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:37.142323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567a20d-8c59-432d-b6ee-4843c42e830d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2567a20d-8c59-432d-b6ee-4843c42e830d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:37.155323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:37.156200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:37.157121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:37.157121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:37.157121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:37.157121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:37.157121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:37.157121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:37.157121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:37.157121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:37.157121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 281/568] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:37.157590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567a20d-8c59-432d-b6ee-4843c42e830d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:37.171707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab18e1-d464-4131-81fd-5a8080589c66 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:37.207404 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f8bdf-7662-494a-aa92-43c6fb7ae8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:37.208037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f8bdf-7662-494a-aa92-43c6fb7ae8dc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:37.208314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f8bdf-7662-494a-aa92-43c6fb7ae8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:37.208586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f8bdf-7662-494a-aa92-43c6fb7ae8dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:37.216522 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f8bdf-7662-494a-aa92-43c6fb7ae8dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:37.218104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f8bdf-7662-494a-aa92-43c6fb7ae8dc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a03f8bdf-7662-494a-aa92-43c6fb7ae8dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:37.229900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:37.230958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:37.231996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:37.231996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:37.231996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:37.231996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:37.231996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:37.231996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:37.231996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:37.231996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:37.231996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:00:37.231996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:37.231996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 288/569] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Sep 6 21:00:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:37.231996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f8bdf-7662-494a-aa92-43c6fb7ae8dc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:37.265531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567a20d-8c59-432d-b6ee-4843c42e830d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:37.299187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6457b2b-b62e-44b3-a42c-193f945a08d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:37.299790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6457b2b-b62e-44b3-a42c-193f945a08d2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:37.300080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6457b2b-b62e-44b3-a42c-193f945a08d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:37.300382 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6457b2b-b62e-44b3-a42c-193f945a08d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:37.310850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6457b2b-b62e-44b3-a42c-193f945a08d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c6457b2b-b62e-44b3-a42c-193f945a08d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:37.315674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:37.316586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:37.317446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:37.317446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:37.317446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:37.317446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:37.317446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:37.317446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:37.317446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:37.317446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:37.317446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:37.317446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:37.317446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:37.317446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:37.317446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 282/570] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:37.318112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6457b2b-b62e-44b3-a42c-193f945a08d2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:37.328863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:37.333116 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f8bdf-7662-494a-aa92-43c6fb7ae8dc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:37.357890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:37.372106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5845cd-b301-4d7e-a564-d15ba6d0383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:37.372753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5845cd-b301-4d7e-a564-d15ba6d0383f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:37.373031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5845cd-b301-4d7e-a564-d15ba6d0383f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:37.373303 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5845cd-b301-4d7e-a564-d15ba6d0383f None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:37.373947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5845cd-b301-4d7e-a564-d15ba6d0383f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:37.374254 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5845cd-b301-4d7e-a564-d15ba6d0383f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:37.375636 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5845cd-b301-4d7e-a564-d15ba6d0383f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:37.387824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 289/571] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:37.388684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5845cd-b301-4d7e-a564-d15ba6d0383f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:37.437805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6457b2b-b62e-44b3-a42c-193f945a08d2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:37.476116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a6482-06b3-41ab-8c15-31845f657d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:37.476741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a6482-06b3-41ab-8c15-31845f657d54 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:37.477039 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a6482-06b3-41ab-8c15-31845f657d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:37.477277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a6482-06b3-41ab-8c15-31845f657d54 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:37.478051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a6482-06b3-41ab-8c15-31845f657d54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:37.478346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a6482-06b3-41ab-8c15-31845f657d54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:37.479805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a6482-06b3-41ab-8c15-31845f657d54 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:37.493040 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 283/572] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:37.493868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a6482-06b3-41ab-8c15-31845f657d54 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:37.510463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5845cd-b301-4d7e-a564-d15ba6d0383f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:37.547609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc4ead-4582-4aff-8451-e24e8e7bcb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:37.548538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc4ead-4582-4aff-8451-e24e8e7bcb10 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:37.549872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc4ead-4582-4aff-8451-e24e8e7bcb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:37.550168 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc4ead-4582-4aff-8451-e24e8e7bcb10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:37.562134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc4ead-4582-4aff-8451-e24e8e7bcb10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f7cc4ead-4582-4aff-8451-e24e8e7bcb10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:37.568282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:37.570027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 290/573] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:37.570985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc4ead-4582-4aff-8451-e24e8e7bcb10 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:37.574107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:37.616359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a6482-06b3-41ab-8c15-31845f657d54 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:37.623913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:37.663848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b71af-e7d5-48b0-b1c4-e9364233ba4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:37.664565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b71af-e7d5-48b0-b1c4-e9364233ba4e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:37.664952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b71af-e7d5-48b0-b1c4-e9364233ba4e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:37.665308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b71af-e7d5-48b0-b1c4-e9364233ba4e None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:37.666481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b71af-e7d5-48b0-b1c4-e9364233ba4e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:37.666993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b71af-e7d5-48b0-b1c4-e9364233ba4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:37.668851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b71af-e7d5-48b0-b1c4-e9364233ba4e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:37.687714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 284/574] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Sep 6 21:00:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:37.688692 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b71af-e7d5-48b0-b1c4-e9364233ba4e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:37.697398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc4ead-4582-4aff-8451-e24e8e7bcb10 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:37.732832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bbcac-5ff7-4356-bf47-df96e456580d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:37.733422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bbcac-5ff7-4356-bf47-df96e456580d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:37.733716 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bbcac-5ff7-4356-bf47-df96e456580d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:37.733987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bbcac-5ff7-4356-bf47-df96e456580d None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:37.734654 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bbcac-5ff7-4356-bf47-df96e456580d None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:37.734955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bbcac-5ff7-4356-bf47-df96e456580d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:37.736314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bbcac-5ff7-4356-bf47-df96e456580d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:37.744516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 291/575] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:37.745392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bbcac-5ff7-4356-bf47-df96e456580d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:37.789604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b71af-e7d5-48b0-b1c4-e9364233ba4e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:37.816617 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:37.827691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99421fd8-ed1b-4b8b-b4d4-7d3dc95aa52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:37.828327 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99421fd8-ed1b-4b8b-b4d4-7d3dc95aa52f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:37.828601 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99421fd8-ed1b-4b8b-b4d4-7d3dc95aa52f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:37.828902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99421fd8-ed1b-4b8b-b4d4-7d3dc95aa52f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:37.836052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99421fd8-ed1b-4b8b-b4d4-7d3dc95aa52f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:37.837926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99421fd8-ed1b-4b8b-b4d4-7d3dc95aa52f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:37.847038 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bbcac-5ff7-4356-bf47-df96e456580d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:37.849393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-99421fd8-ed1b-4b8b-b4d4-7d3dc95aa52f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:37.849519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:37.850428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:37.851357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:37.851357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:37.851357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:37.851357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:37.851357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:37.851357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:37.851357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:37.851357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:37.851357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 21:00:37.851357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:37.851357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 285/576] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:37.852636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99421fd8-ed1b-4b8b-b4d4-7d3dc95aa52f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:37.882363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf79b15-6c39-4965-acff-eb4645426959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:37.882974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf79b15-6c39-4965-acff-eb4645426959 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:37.883244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf79b15-6c39-4965-acff-eb4645426959 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:37.883571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf79b15-6c39-4965-acff-eb4645426959 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:37.884516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf79b15-6c39-4965-acff-eb4645426959 None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=061c584de8264868af8782402a98a72e) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:37.884799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf79b15-6c39-4965-acff-eb4645426959 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:37.886250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf79b15-6c39-4965-acff-eb4645426959 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:37.891796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 292/577] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:37 2022] GET /identity/v3/role_assignments?user.id=061c584de8264868af8782402a98a72e&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:37.892607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf79b15-6c39-4965-acff-eb4645426959 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:37.973727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99421fd8-ed1b-4b8b-b4d4-7d3dc95aa52f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:38.011033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e03c7-d12c-4c3f-a8ef-403798eff01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:38.011760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e03c7-d12c-4c3f-a8ef-403798eff01d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:38.012107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e03c7-d12c-4c3f-a8ef-403798eff01d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:38.012519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e03c7-d12c-4c3f-a8ef-403798eff01d None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:38.013294 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e03c7-d12c-4c3f-a8ef-403798eff01d None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:38.013671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e03c7-d12c-4c3f-a8ef-403798eff01d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:38.015100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e03c7-d12c-4c3f-a8ef-403798eff01d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:38.016775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf79b15-6c39-4965-acff-eb4645426959 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:38.029097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 286/578] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:38.029097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e03c7-d12c-4c3f-a8ef-403798eff01d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:38.061005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3eb84-16e1-4feb-9468-f947abe76b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:38.061761 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3eb84-16e1-4feb-9468-f947abe76b57 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:38.062047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3eb84-16e1-4feb-9468-f947abe76b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:38.062317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3eb84-16e1-4feb-9468-f947abe76b57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:38.073972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3eb84-16e1-4feb-9468-f947abe76b57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:38.075627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3eb84-16e1-4feb-9468-f947abe76b57 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:38.083396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-12c3eb84-16e1-4feb-9468-f947abe76b57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:38.091683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:38.092804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:38.093687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:38.093687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:38.093687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:38.093687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 293/579] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:38.094349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3eb84-16e1-4feb-9468-f947abe76b57 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:38.116800 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:38.148645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e03c7-d12c-4c3f-a8ef-403798eff01d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:38.186330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c796e4-a050-4370-97f1-c25586bf1bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:38.187065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c796e4-a050-4370-97f1-c25586bf1bea None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:38.187408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c796e4-a050-4370-97f1-c25586bf1bea None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:38.187747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c796e4-a050-4370-97f1-c25586bf1bea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:38.199089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c796e4-a050-4370-97f1-c25586bf1bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:38.200600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3eb84-16e1-4feb-9468-f947abe76b57 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:38.200737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c796e4-a050-4370-97f1-c25586bf1bea None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-65c796e4-a050-4370-97f1-c25586bf1bea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:38.216228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:38.217283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:38.218221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:38.218221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:38.218221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:38.218221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 287/580] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:38.222216 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c796e4-a050-4370-97f1-c25586bf1bea None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:38.245420 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997afed-3039-4b5f-84d2-a2121b4e8da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:38.246426 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997afed-3039-4b5f-84d2-a2121b4e8da9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:38.246943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997afed-3039-4b5f-84d2-a2121b4e8da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:38.247439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997afed-3039-4b5f-84d2-a2121b4e8da9 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:38.248459 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997afed-3039-4b5f-84d2-a2121b4e8da9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:38.248795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997afed-3039-4b5f-84d2-a2121b4e8da9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:38.250319 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997afed-3039-4b5f-84d2-a2121b4e8da9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:38.274534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 294/581] 10.222.0.79 () {56 vars in 1137 bytes} [Tue Sep 6 21:00:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:38.274534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997afed-3039-4b5f-84d2-a2121b4e8da9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:38.338590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:38.342861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c796e4-a050-4370-97f1-c25586bf1bea None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:38.357000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:38.382130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997afed-3039-4b5f-84d2-a2121b4e8da9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:38.383112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91b7f7-ebe1-4da4-8373-5e7ee43377de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:38.383759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91b7f7-ebe1-4da4-8373-5e7ee43377de None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:38.384036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91b7f7-ebe1-4da4-8373-5e7ee43377de None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:38.384325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91b7f7-ebe1-4da4-8373-5e7ee43377de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:38.395107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91b7f7-ebe1-4da4-8373-5e7ee43377de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:38.396571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91b7f7-ebe1-4da4-8373-5e7ee43377de None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ab91b7f7-ebe1-4da4-8373-5e7ee43377de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:38.411070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:38.412024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:38.413255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:38.413255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:38.413255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:38.413255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 288/582] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:38.413255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91b7f7-ebe1-4da4-8373-5e7ee43377de None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:38.424735 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11090e-f25e-483a-adef-1ad783a7d1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:38.425356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11090e-f25e-483a-adef-1ad783a7d1fc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:38.425636 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11090e-f25e-483a-adef-1ad783a7d1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:38.425903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11090e-f25e-483a-adef-1ad783a7d1fc None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:38.426546 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11090e-f25e-483a-adef-1ad783a7d1fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:38.426853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11090e-f25e-483a-adef-1ad783a7d1fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:38.428230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11090e-f25e-483a-adef-1ad783a7d1fc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:38.442017 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 295/583] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:38.443081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11090e-f25e-483a-adef-1ad783a7d1fc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:38.535753 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91b7f7-ebe1-4da4-8373-5e7ee43377de None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:38.549390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11090e-f25e-483a-adef-1ad783a7d1fc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:38.571063 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262fc208-af0d-4143-9af8-59fdfefc3b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:38.571683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262fc208-af0d-4143-9af8-59fdfefc3b13 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:38.572010 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262fc208-af0d-4143-9af8-59fdfefc3b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:38.572290 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262fc208-af0d-4143-9af8-59fdfefc3b13 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:38.572969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262fc208-af0d-4143-9af8-59fdfefc3b13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:38.573272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262fc208-af0d-4143-9af8-59fdfefc3b13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:38.574668 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262fc208-af0d-4143-9af8-59fdfefc3b13 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:38.586969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 289/584] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:38.587825 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262fc208-af0d-4143-9af8-59fdfefc3b13 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:38.598262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c1c48-559a-4a19-b586-68b552b3e87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:38.598922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c1c48-559a-4a19-b586-68b552b3e87a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:38.599197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c1c48-559a-4a19-b586-68b552b3e87a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:38.599453 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c1c48-559a-4a19-b586-68b552b3e87a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:38.603623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:38.610495 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:38.615084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c1c48-559a-4a19-b586-68b552b3e87a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:38.616751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c1c48-559a-4a19-b586-68b552b3e87a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-100c1c48-559a-4a19-b586-68b552b3e87a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:38.632313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:38.633370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:38.634328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:38.634328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:38.634328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:38.634328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 296/585] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:38.634328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c1c48-559a-4a19-b586-68b552b3e87a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:38.707078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262fc208-af0d-4143-9af8-59fdfefc3b13 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:38.741875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c1c48-559a-4a19-b586-68b552b3e87a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:38.745520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab86b69-9641-4447-9ff7-854fdc80da10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:38.746235 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab86b69-9641-4447-9ff7-854fdc80da10 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:38.746520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab86b69-9641-4447-9ff7-854fdc80da10 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:38.746830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab86b69-9641-4447-9ff7-854fdc80da10 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:38.747478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab86b69-9641-4447-9ff7-854fdc80da10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:38.747782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab86b69-9641-4447-9ff7-854fdc80da10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:38.750387 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab86b69-9641-4447-9ff7-854fdc80da10 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:38.768281 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 290/586] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:38.768281 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab86b69-9641-4447-9ff7-854fdc80da10 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:38.784985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cbcda-4b08-4f8c-8c7d-4de92f7ddc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:38.785640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cbcda-4b08-4f8c-8c7d-4de92f7ddc9e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:38.785936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cbcda-4b08-4f8c-8c7d-4de92f7ddc9e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:38.786212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cbcda-4b08-4f8c-8c7d-4de92f7ddc9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:38.794054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cbcda-4b08-4f8c-8c7d-4de92f7ddc9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:38.795619 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cbcda-4b08-4f8c-8c7d-4de92f7ddc9e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ac8cbcda-4b08-4f8c-8c7d-4de92f7ddc9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:38.807817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:38.808755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:38.809656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:38.809656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:38.809656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:38.809656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:38.809656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:38.809656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:38.809656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:38.809656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:38.809656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 297/587] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:38.810156 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cbcda-4b08-4f8c-8c7d-4de92f7ddc9e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:38.847728 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:38.866834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:38.882642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab86b69-9641-4447-9ff7-854fdc80da10 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:38.917880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ad371-7220-4050-9f03-f5466b01f24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:38.918716 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ad371-7220-4050-9f03-f5466b01f24e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:38.919089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ad371-7220-4050-9f03-f5466b01f24e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:38.919450 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ad371-7220-4050-9f03-f5466b01f24e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:38.924290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cbcda-4b08-4f8c-8c7d-4de92f7ddc9e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:38.928255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ad371-7220-4050-9f03-f5466b01f24e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:38.929854 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ad371-7220-4050-9f03-f5466b01f24e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b57ad371-7220-4050-9f03-f5466b01f24e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:38.942580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:38.943568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:38.945252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:38.945252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:38.945252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:38.945252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:38.945252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:38.945252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:38.945252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:38.945252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:38.945252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 21:00:38.945252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:38.945252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 291/588] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:38.945252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ad371-7220-4050-9f03-f5466b01f24e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:38.969307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a02c6d-3cdb-42ae-909b-6fa0faa30de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:38.969945 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a02c6d-3cdb-42ae-909b-6fa0faa30de5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:38.970191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a02c6d-3cdb-42ae-909b-6fa0faa30de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:38.970447 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a02c6d-3cdb-42ae-909b-6fa0faa30de5 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:38.971094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a02c6d-3cdb-42ae-909b-6fa0faa30de5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:38.971383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a02c6d-3cdb-42ae-909b-6fa0faa30de5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:38.972781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a02c6d-3cdb-42ae-909b-6fa0faa30de5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:38.985253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 298/589] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:38.986471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a02c6d-3cdb-42ae-909b-6fa0faa30de5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:39.060920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ad371-7220-4050-9f03-f5466b01f24e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:39.096025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:39.102347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a02c6d-3cdb-42ae-909b-6fa0faa30de5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:39.102855 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969af33d-468b-4476-b887-24784319a32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:39.103092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969af33d-468b-4476-b887-24784319a32f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:39.103369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969af33d-468b-4476-b887-24784319a32f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:39.103638 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969af33d-468b-4476-b887-24784319a32f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:39.111984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969af33d-468b-4476-b887-24784319a32f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:39.113490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969af33d-468b-4476-b887-24784319a32f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:39.115408 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-969af33d-468b-4476-b887-24784319a32f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:39.125955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:39.126972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:39.134335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:39.134335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:39.134335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:39.134335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:39.134335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:39.134335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:39.134335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:39.134335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:39.134335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 21:00:39.134335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:39.134335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 292/590] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:39.134335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969af33d-468b-4476-b887-24784319a32f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:39.148476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3db32-3ad9-489e-8146-18f4b9bb6639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:39.149121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3db32-3ad9-489e-8146-18f4b9bb6639 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:39.149444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3db32-3ad9-489e-8146-18f4b9bb6639 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:39.149748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3db32-3ad9-489e-8146-18f4b9bb6639 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:39.150507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3db32-3ad9-489e-8146-18f4b9bb6639 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:39.150814 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3db32-3ad9-489e-8146-18f4b9bb6639 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:39.152456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3db32-3ad9-489e-8146-18f4b9bb6639 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:39.165158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 299/591] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:39.166455 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3db32-3ad9-489e-8146-18f4b9bb6639 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:39.248773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969af33d-468b-4476-b887-24784319a32f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:39.292443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3db32-3ad9-489e-8146-18f4b9bb6639 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:39.295676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec2b30-6493-4dfe-ad72-bb2579ff5124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:39.296301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec2b30-6493-4dfe-ad72-bb2579ff5124 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:39.296574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec2b30-6493-4dfe-ad72-bb2579ff5124 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:39.296873 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec2b30-6493-4dfe-ad72-bb2579ff5124 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:39.297682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec2b30-6493-4dfe-ad72-bb2579ff5124 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:39.297982 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec2b30-6493-4dfe-ad72-bb2579ff5124 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:39.299433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec2b30-6493-4dfe-ad72-bb2579ff5124 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:39.319621 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 293/592] 10.222.0.79 () {56 vars in 1137 bytes} [Tue Sep 6 21:00:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:39.325036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec2b30-6493-4dfe-ad72-bb2579ff5124 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:39.330438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b009634-3ca8-475a-83e7-fe73bbcb94c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:39.331088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b009634-3ca8-475a-83e7-fe73bbcb94c3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:39.331368 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b009634-3ca8-475a-83e7-fe73bbcb94c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:39.331638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b009634-3ca8-475a-83e7-fe73bbcb94c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:39.338948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b009634-3ca8-475a-83e7-fe73bbcb94c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:39.340452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b009634-3ca8-475a-83e7-fe73bbcb94c3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2b009634-3ca8-475a-83e7-fe73bbcb94c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:39.354832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:39.355785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:39.357389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:39.357389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:39.357389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:39.357389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:39.357389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:39.357389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:39.357389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:39.357389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:39.357389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 300/593] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:39.357389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b009634-3ca8-475a-83e7-fe73bbcb94c3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:39.391983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:39.397752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:39.465799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec2b30-6493-4dfe-ad72-bb2579ff5124 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:39.479528 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b009634-3ca8-475a-83e7-fe73bbcb94c3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:39.510451 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f273b-73b0-44b6-92e1-8dd8ec3bf011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:39.511069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f273b-73b0-44b6-92e1-8dd8ec3bf011 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:39.511346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f273b-73b0-44b6-92e1-8dd8ec3bf011 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:39.511619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f273b-73b0-44b6-92e1-8dd8ec3bf011 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:39.512455 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f273b-73b0-44b6-92e1-8dd8ec3bf011 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:39.512839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f273b-73b0-44b6-92e1-8dd8ec3bf011 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:39.514323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f273b-73b0-44b6-92e1-8dd8ec3bf011 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:39.515646 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f42be-8670-4aa7-93fa-cad935868a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:39.516319 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f42be-8670-4aa7-93fa-cad935868a64 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:39.516601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f42be-8670-4aa7-93fa-cad935868a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:39.516982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f42be-8670-4aa7-93fa-cad935868a64 None admin] PATH_INFO: `/v3/projects/05af249ba0a047fe83aefb078e203e53/users/12d51454d20c46d7a5e6d963a2f987cc/roles/b3f49281e89f4fa1a2d119e2fb4922e5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:39.533833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f42be-8670-4aa7-93fa-cad935868a64 None admin] RBAC: Authorizing `identity:create_grant(project_id=05af249ba0a047fe83aefb078e203e53, user_id=12d51454d20c46d7a5e6d963a2f987cc, role_id=b3f49281e89f4fa1a2d119e2fb4922e5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:39.535312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f42be-8670-4aa7-93fa-cad935868a64 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:39.538397 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 294/594] 10.222.0.79 () {56 vars in 1139 bytes} [Tue Sep 6 21:00:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:39.539249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f273b-73b0-44b6-92e1-8dd8ec3bf011 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:39.562956 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 301/595] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:00:39 2022] PUT /identity/v3/projects/05af249ba0a047fe83aefb078e203e53/users/12d51454d20c46d7a5e6d963a2f987cc/roles/b3f49281e89f4fa1a2d119e2fb4922e5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:00:39.564597 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f42be-8670-4aa7-93fa-cad935868a64 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:39.677428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f273b-73b0-44b6-92e1-8dd8ec3bf011 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:39.690178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:39.719517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:39.723220 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401cad7-2029-4d4e-8986-6ec114a59cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:39.724178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401cad7-2029-4d4e-8986-6ec114a59cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:39.724623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401cad7-2029-4d4e-8986-6ec114a59cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:39.725948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401cad7-2029-4d4e-8986-6ec114a59cf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:39.733500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f42be-8670-4aa7-93fa-cad935868a64 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:39.739869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401cad7-2029-4d4e-8986-6ec114a59cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:39.741709 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401cad7-2029-4d4e-8986-6ec114a59cf5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5401cad7-2029-4d4e-8986-6ec114a59cf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:39.759550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:39.760782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:39.761794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:39.761794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:39.761794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:39.763768 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 295/596] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:39.764931 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401cad7-2029-4d4e-8986-6ec114a59cf5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:39.775267 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e23d8-23bb-418b-b677-f5e58941f9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:39.776316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e23d8-23bb-418b-b677-f5e58941f9eb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:39.776316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e23d8-23bb-418b-b677-f5e58941f9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:39.776541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e23d8-23bb-418b-b677-f5e58941f9eb None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:39.777192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e23d8-23bb-418b-b677-f5e58941f9eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:39.777499 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e23d8-23bb-418b-b677-f5e58941f9eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:39.779049 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e23d8-23bb-418b-b677-f5e58941f9eb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:39.794277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 302/597] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:39.795074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e23d8-23bb-418b-b677-f5e58941f9eb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:39.904559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401cad7-2029-4d4e-8986-6ec114a59cf5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:39.925788 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e23d8-23bb-418b-b677-f5e58941f9eb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:39.948242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb1646-6de2-4d8b-9839-091a9f0f1167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:39.949057 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb1646-6de2-4d8b-9839-091a9f0f1167 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:39.949374 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb1646-6de2-4d8b-9839-091a9f0f1167 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:39.949688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb1646-6de2-4d8b-9839-091a9f0f1167 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:39.966467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb1646-6de2-4d8b-9839-091a9f0f1167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:39.968116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb1646-6de2-4d8b-9839-091a9f0f1167 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:39.972231 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c7fc7-6890-448a-b30b-d1900761a718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:39.972912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c7fc7-6890-448a-b30b-d1900761a718 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:39.973194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c7fc7-6890-448a-b30b-d1900761a718 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:39.973467 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c7fc7-6890-448a-b30b-d1900761a718 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:39.974143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c7fc7-6890-448a-b30b-d1900761a718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:39.974444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c7fc7-6890-448a-b30b-d1900761a718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:39.976804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c7fc7-6890-448a-b30b-d1900761a718 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b4fb1646-6de2-4d8b-9839-091a9f0f1167 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:39.985966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:39.987043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:39.987900 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:39.987900 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:39.987900 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:39.987900 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 296/598] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:39.988130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:39.989627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb1646-6de2-4d8b-9839-091a9f0f1167 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:39.992340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 303/599] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:40.001772 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c7fc7-6890-448a-b30b-d1900761a718 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:40.010239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:40.129589 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c7fc7-6890-448a-b30b-d1900761a718 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:40.143613 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb1646-6de2-4d8b-9839-091a9f0f1167 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:40.170505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb7ab2-bcce-4ef0-ae00-1c84769a0e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:40.171232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb7ab2-bcce-4ef0-ae00-1c84769a0e91 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:40.171509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb7ab2-bcce-4ef0-ae00-1c84769a0e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:40.171885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb7ab2-bcce-4ef0-ae00-1c84769a0e91 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:40.172527 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb7ab2-bcce-4ef0-ae00-1c84769a0e91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:40.172853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb7ab2-bcce-4ef0-ae00-1c84769a0e91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:40.174354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb7ab2-bcce-4ef0-ae00-1c84769a0e91 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:40.179790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b6f2a-816a-4afe-9122-44e6565b93e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:40.180557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b6f2a-816a-4afe-9122-44e6565b93e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:40.181003 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b6f2a-816a-4afe-9122-44e6565b93e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:40.181244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b6f2a-816a-4afe-9122-44e6565b93e8 None admin] PATH_INFO: `/v3/projects/05af249ba0a047fe83aefb078e203e53/users/a76937ab52e445c59b9eb9638fd4a753/roles/b3f49281e89f4fa1a2d119e2fb4922e5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:40.192730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 304/600] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:40.192730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb7ab2-bcce-4ef0-ae00-1c84769a0e91 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:40.201237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b6f2a-816a-4afe-9122-44e6565b93e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=05af249ba0a047fe83aefb078e203e53, user_id=a76937ab52e445c59b9eb9638fd4a753, role_id=b3f49281e89f4fa1a2d119e2fb4922e5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:40.202643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b6f2a-816a-4afe-9122-44e6565b93e8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:40.218455 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 297/601] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:00:39 2022] PUT /identity/v3/projects/05af249ba0a047fe83aefb078e203e53/users/a76937ab52e445c59b9eb9638fd4a753/roles/b3f49281e89f4fa1a2d119e2fb4922e5 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:00:40.222085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb4eeb-462e-40d7-8d9c-ae3abc8b85e1 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:40.222654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb4eeb-462e-40d7-8d9c-ae3abc8b85e1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:40.223143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb4eeb-462e-40d7-8d9c-ae3abc8b85e1 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:40.232221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 298/602] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:40 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:40.233658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb4eeb-462e-40d7-8d9c-ae3abc8b85e1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:40.279903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:40.330915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb7ab2-bcce-4ef0-ae00-1c84769a0e91 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:40.332755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:40.380776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3c4bc-9606-47fb-aca5-0d83ad1f7a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:40.381780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3c4bc-9606-47fb-aca5-0d83ad1f7a69 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:40.382459 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3c4bc-9606-47fb-aca5-0d83ad1f7a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:40.383317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3c4bc-9606-47fb-aca5-0d83ad1f7a69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:40.387240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb4eeb-462e-40d7-8d9c-ae3abc8b85e1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:40.394493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3c4bc-9606-47fb-aca5-0d83ad1f7a69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:40.397127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3c4bc-9606-47fb-aca5-0d83ad1f7a69 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-66b3c4bc-9606-47fb-aca5-0d83ad1f7a69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:40.414072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:40.415084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:40.416277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:40.416277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:40.416277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:40.416277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:40.416277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:40.416277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:40.416277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:40.416277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:40.419445 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 305/603] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:40.421139 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512db4af-9fef-4ff8-a5f6-ba46c00593c5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:40.421911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512db4af-9fef-4ff8-a5f6-ba46c00593c5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:40.422345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512db4af-9fef-4ff8-a5f6-ba46c00593c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:40.430328 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772bd9f-efa7-4a4e-8642-e9eae5ccd400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:40.431439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772bd9f-efa7-4a4e-8642-e9eae5ccd400 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:40.431575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772bd9f-efa7-4a4e-8642-e9eae5ccd400 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:40.431914 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772bd9f-efa7-4a4e-8642-e9eae5ccd400 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:40.441531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772bd9f-efa7-4a4e-8642-e9eae5ccd400 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:40.443206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772bd9f-efa7-4a4e-8642-e9eae5ccd400 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:40.443633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-512db4af-9fef-4ff8-a5f6-ba46c00593c5 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:40.444876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-512db4af-9fef-4ff8-a5f6-ba46c00593c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8772bd9f-efa7-4a4e-8642-e9eae5ccd400 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:40.456600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:40.457809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:40.458878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:40.458878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:40.458878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:40.458878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:40.458878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:40.458878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:40.458878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:40.458878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:40.459966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 299/604] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:40.460547 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772bd9f-efa7-4a4e-8642-e9eae5ccd400 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:40.548551 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 306/605] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:40.549580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512db4af-9fef-4ff8-a5f6-ba46c00593c5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:40.620028 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772bd9f-efa7-4a4e-8642-e9eae5ccd400 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:40.624690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:40.664985 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:40.682665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512db4af-9fef-4ff8-a5f6-ba46c00593c5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:40.686266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a1142-7a39-4820-a95f-9171b15703a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:40.686947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a1142-7a39-4820-a95f-9171b15703a9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:40.687285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a1142-7a39-4820-a95f-9171b15703a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:40.687616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a1142-7a39-4820-a95f-9171b15703a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:40.688375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a1142-7a39-4820-a95f-9171b15703a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:40.688759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a1142-7a39-4820-a95f-9171b15703a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:40.690264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a1142-7a39-4820-a95f-9171b15703a9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:40.714355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 300/606] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:40.715496 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a1142-7a39-4820-a95f-9171b15703a9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:40.722112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26969a4-daaa-484d-9d78-54f2e5c2d019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:40.722728 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26969a4-daaa-484d-9d78-54f2e5c2d019 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:40.723003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26969a4-daaa-484d-9d78-54f2e5c2d019 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:40.723273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26969a4-daaa-484d-9d78-54f2e5c2d019 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:40.723967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26969a4-daaa-484d-9d78-54f2e5c2d019 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:40.724267 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26969a4-daaa-484d-9d78-54f2e5c2d019 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:40.726629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26969a4-daaa-484d-9d78-54f2e5c2d019 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:40.752735 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 307/607] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:40.758712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26969a4-daaa-484d-9d78-54f2e5c2d019 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:40.859197 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a1142-7a39-4820-a95f-9171b15703a9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:40.897077 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a157f-aaf7-4a0c-bc6d-cbdc0b012954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:40.898009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a157f-aaf7-4a0c-bc6d-cbdc0b012954 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:40.898430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a157f-aaf7-4a0c-bc6d-cbdc0b012954 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:40.898842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a157f-aaf7-4a0c-bc6d-cbdc0b012954 None admin] PATH_INFO: `/v3/projects/05af249ba0a047fe83aefb078e203e53/users/f50ae3b9997d4dc39c3d67ca40bbc566/roles/b3f49281e89f4fa1a2d119e2fb4922e5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:40.914859 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a157f-aaf7-4a0c-bc6d-cbdc0b012954 None admin] RBAC: Authorizing `identity:create_grant(project_id=05af249ba0a047fe83aefb078e203e53, user_id=f50ae3b9997d4dc39c3d67ca40bbc566, role_id=b3f49281e89f4fa1a2d119e2fb4922e5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:40.917007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a157f-aaf7-4a0c-bc6d-cbdc0b012954 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:40.932780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 301/608] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:00:40 2022] PUT /identity/v3/projects/05af249ba0a047fe83aefb078e203e53/users/f50ae3b9997d4dc39c3d67ca40bbc566/roles/b3f49281e89f4fa1a2d119e2fb4922e5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:00:40.947208 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a157f-aaf7-4a0c-bc6d-cbdc0b012954 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:40.985211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26969a4-daaa-484d-9d78-54f2e5c2d019 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:41.006424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:41.018648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:41.098021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a157f-aaf7-4a0c-bc6d-cbdc0b012954 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:41.105454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93f9e9-3ac6-4c65-9fbb-ccbe4348d382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:41.106215 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93f9e9-3ac6-4c65-9fbb-ccbe4348d382 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:41.106499 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93f9e9-3ac6-4c65-9fbb-ccbe4348d382 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:41.106780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93f9e9-3ac6-4c65-9fbb-ccbe4348d382 None admin] PATH_INFO: `/v3/projects/05af249ba0a047fe83aefb078e203e53/users/d5a45dec40844877a6b5b301fc44a110/roles/b3f49281e89f4fa1a2d119e2fb4922e5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:41.137426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0311e-693e-447f-9961-a969c9bd6da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:41.138342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0311e-693e-447f-9961-a969c9bd6da1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:41.138910 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0311e-693e-447f-9961-a969c9bd6da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:41.139340 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0311e-693e-447f-9961-a969c9bd6da1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:41.148577 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93f9e9-3ac6-4c65-9fbb-ccbe4348d382 None admin] RBAC: Authorizing `identity:create_grant(project_id=05af249ba0a047fe83aefb078e203e53, user_id=d5a45dec40844877a6b5b301fc44a110, role_id=b3f49281e89f4fa1a2d119e2fb4922e5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:41.150194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93f9e9-3ac6-4c65-9fbb-ccbe4348d382 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:41.152245 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0311e-693e-447f-9961-a969c9bd6da1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-54f0311e-693e-447f-9961-a969c9bd6da1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:41.158093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:41.159287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:41.160332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:41.160332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:41.160332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:41.160332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:41.160332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:41.160332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:41.160332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:41.160332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:41.160332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:41.160332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:41.160332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:41.160332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:41.163203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 302/609] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:00:41.169703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0311e-693e-447f-9961-a969c9bd6da1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:41.191705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 308/610] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:00:40 2022] PUT /identity/v3/projects/05af249ba0a047fe83aefb078e203e53/users/d5a45dec40844877a6b5b301fc44a110/roles/b3f49281e89f4fa1a2d119e2fb4922e5 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:00:41.301378 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:41.311224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0311e-693e-447f-9961-a969c9bd6da1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:41.360571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994c020-db47-4a35-aa6a-94326baccc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:41.361218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994c020-db47-4a35-aa6a-94326baccc26 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:41.361490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994c020-db47-4a35-aa6a-94326baccc26 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:41.361754 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994c020-db47-4a35-aa6a-94326baccc26 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:41.362434 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994c020-db47-4a35-aa6a-94326baccc26 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:41.362726 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994c020-db47-4a35-aa6a-94326baccc26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:41.364138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994c020-db47-4a35-aa6a-94326baccc26 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:41.376487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 303/611] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:41.382325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93f9e9-3ac6-4c65-9fbb-ccbe4348d382 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:41.511170 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93f9e9-3ac6-4c65-9fbb-ccbe4348d382 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:41.548143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeee7f9-7814-4d7b-9d84-f6ca4173f6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:41.548821 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeee7f9-7814-4d7b-9d84-f6ca4173f6fe None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:41.549098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeee7f9-7814-4d7b-9d84-f6ca4173f6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:41.549367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeee7f9-7814-4d7b-9d84-f6ca4173f6fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:41.561316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeee7f9-7814-4d7b-9d84-f6ca4173f6fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:41.562714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeee7f9-7814-4d7b-9d84-f6ca4173f6fe None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-efeee7f9-7814-4d7b-9d84-f6ca4173f6fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:41.578751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:41.579828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:41.580724 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:41.580724 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:41.580724 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:41.580724 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 309/612] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:41.586942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994c020-db47-4a35-aa6a-94326baccc26 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:41.777394 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994c020-db47-4a35-aa6a-94326baccc26 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:41.826874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb34ab-0ff5-4fa6-949d-bb15f1b3c946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:41.827411 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb34ab-0ff5-4fa6-949d-bb15f1b3c946 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:41.827684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb34ab-0ff5-4fa6-949d-bb15f1b3c946 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:41.827952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb34ab-0ff5-4fa6-949d-bb15f1b3c946 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:41.828605 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb34ab-0ff5-4fa6-949d-bb15f1b3c946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:41.828927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb34ab-0ff5-4fa6-949d-bb15f1b3c946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:41.830311 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb34ab-0ff5-4fa6-949d-bb15f1b3c946 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:41.852160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 304/613] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:41.858320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeee7f9-7814-4d7b-9d84-f6ca4173f6fe None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:41.882405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:41.995765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeee7f9-7814-4d7b-9d84-f6ca4173f6fe None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:42.038336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba5b7f-c4c9-4a7a-86b9-421bdc5b5b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:42.039359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba5b7f-c4c9-4a7a-86b9-421bdc5b5b95 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:42.039800 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba5b7f-c4c9-4a7a-86b9-421bdc5b5b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:42.040257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba5b7f-c4c9-4a7a-86b9-421bdc5b5b95 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:42.050680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba5b7f-c4c9-4a7a-86b9-421bdc5b5b95 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:42.052583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba5b7f-c4c9-4a7a-86b9-421bdc5b5b95 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f9ba5b7f-c4c9-4a7a-86b9-421bdc5b5b95 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:42.069840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:42.070893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:42.071968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:42.071968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:42.071968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:42.071968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:42.071968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:42.071968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:42.071968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:42.071968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:42.071968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 21:00:42.071968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:42.071968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 310/614] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:00:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:42.077469 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb34ab-0ff5-4fa6-949d-bb15f1b3c946 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:42.111296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:42.227943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ab8ae-e1b1-4366-97cd-c5032c8b7a75 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:42.228800 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ab8ae-e1b1-4366-97cd-c5032c8b7a75 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:42.228800 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ab8ae-e1b1-4366-97cd-c5032c8b7a75 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:42.230534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 311/615] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:42.240991 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225b038-ad5f-4379-8112-d73a825c0231 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:42.241277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225b038-ad5f-4379-8112-d73a825c0231 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:42.241590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225b038-ad5f-4379-8112-d73a825c0231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:42.242378 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb34ab-0ff5-4fa6-949d-bb15f1b3c946 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:42.280745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6225b038-ad5f-4379-8112-d73a825c0231 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:42.281643 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6225b038-ad5f-4379-8112-d73a825c0231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:42.308980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197441c-ca09-46aa-83a1-e5f368cbc2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:42.310947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197441c-ca09-46aa-83a1-e5f368cbc2ee None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:42.311744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197441c-ca09-46aa-83a1-e5f368cbc2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:42.312504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197441c-ca09-46aa-83a1-e5f368cbc2ee None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:42.314318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197441c-ca09-46aa-83a1-e5f368cbc2ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:42.315166 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197441c-ca09-46aa-83a1-e5f368cbc2ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:42.318323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197441c-ca09-46aa-83a1-e5f368cbc2ee None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:42.360374 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 305/616] 10.222.0.79 () {56 vars in 1135 bytes} [Tue Sep 6 21:00:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:42.362802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197441c-ca09-46aa-83a1-e5f368cbc2ee None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:42.422776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 312/617] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:42.504336 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:42.535873 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197441c-ca09-46aa-83a1-e5f368cbc2ee None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:42.574065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824e594-9d58-460c-98d8-1509508fa78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:42.575182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824e594-9d58-460c-98d8-1509508fa78d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:42.575768 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824e594-9d58-460c-98d8-1509508fa78d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:42.576238 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824e594-9d58-460c-98d8-1509508fa78d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:42.588914 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824e594-9d58-460c-98d8-1509508fa78d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:42.590939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824e594-9d58-460c-98d8-1509508fa78d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1824e594-9d58-460c-98d8-1509508fa78d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:42.617257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:42.618226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:42.619185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:42.619185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:42.619185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:42.619821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 306/618] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:00:42.624925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225b038-ad5f-4379-8112-d73a825c0231 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:42.629769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:42.771926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7f005-f9d8-42f5-ad08-6d1072cb1638 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:42.772338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7f005-f9d8-42f5-ad08-6d1072cb1638 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:42.772457 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7f005-f9d8-42f5-ad08-6d1072cb1638 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:42.775294 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 307/619] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:42.776919 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225b038-ad5f-4379-8112-d73a825c0231 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:42.779673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1cab5-9420-4403-ae5e-777c364b0239 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:42.779986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1cab5-9420-4403-ae5e-777c364b0239 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:42.781057 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1cab5-9420-4403-ae5e-777c364b0239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:42.822333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35393f-9113-408c-b2b3-6ec38f8072fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:42.823348 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35393f-9113-408c-b2b3-6ec38f8072fa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:42.823885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35393f-9113-408c-b2b3-6ec38f8072fa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:42.824528 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35393f-9113-408c-b2b3-6ec38f8072fa None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:42.825438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35393f-9113-408c-b2b3-6ec38f8072fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:42.825930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35393f-9113-408c-b2b3-6ec38f8072fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:42.828136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35393f-9113-408c-b2b3-6ec38f8072fa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:42.833824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-39a1cab5-9420-4403-ae5e-777c364b0239 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:42.835084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-39a1cab5-9420-4403-ae5e-777c364b0239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:42.844643 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 313/620] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:42.865643 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35393f-9113-408c-b2b3-6ec38f8072fa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:42.925347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 308/621] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:42.926154 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1cab5-9420-4403-ae5e-777c364b0239 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:42.949868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:42.989629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:43.032710 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35393f-9113-408c-b2b3-6ec38f8072fa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:43.037198 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1cab5-9420-4403-ae5e-777c364b0239 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:43.074432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d357a-247e-4932-ad2d-5e9e0c0b15c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:43.075156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d357a-247e-4932-ad2d-5e9e0c0b15c7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:43.075430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d357a-247e-4932-ad2d-5e9e0c0b15c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:43.075698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d357a-247e-4932-ad2d-5e9e0c0b15c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:43.087112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d357a-247e-4932-ad2d-5e9e0c0b15c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c08d357a-247e-4932-ad2d-5e9e0c0b15c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:43.092896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:43.094426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:43.095938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:43.095938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:43.095938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:43.095938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:43.095938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:43.095938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:43.095938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:43.095938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:43.095938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:43.095938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:43.095938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:43.095938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:43.095938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 309/622] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:43.100807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d357a-247e-4932-ad2d-5e9e0c0b15c7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:43.144548 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593866b-eb64-4d03-b3a8-aaeee5eb85b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:43.145284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593866b-eb64-4d03-b3a8-aaeee5eb85b6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:43.145560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593866b-eb64-4d03-b3a8-aaeee5eb85b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:43.145827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593866b-eb64-4d03-b3a8-aaeee5eb85b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:43.164582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593866b-eb64-4d03-b3a8-aaeee5eb85b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:43.166275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593866b-eb64-4d03-b3a8-aaeee5eb85b6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8593866b-eb64-4d03-b3a8-aaeee5eb85b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:43.193714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:43.194760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:43.195765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:43.195765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:43.195765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:43.195765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:43.195765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:43.195765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:43.195765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:43.195765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:43.196307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 314/623] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:00:43.200921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593866b-eb64-4d03-b3a8-aaeee5eb85b6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:43.227032 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d357a-247e-4932-ad2d-5e9e0c0b15c7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:43.265030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:43.317372 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f9a37-f7cd-4a92-aebf-43b10fa1324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:43.317995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f9a37-f7cd-4a92-aebf-43b10fa1324d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:43.318270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f9a37-f7cd-4a92-aebf-43b10fa1324d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:43.318533 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f9a37-f7cd-4a92-aebf-43b10fa1324d None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:43.319278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f9a37-f7cd-4a92-aebf-43b10fa1324d None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:43.319631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f9a37-f7cd-4a92-aebf-43b10fa1324d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:43.321277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f9a37-f7cd-4a92-aebf-43b10fa1324d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:43.322513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593866b-eb64-4d03-b3a8-aaeee5eb85b6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:43.340160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 310/624] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:43.350489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:43.356612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f9a37-f7cd-4a92-aebf-43b10fa1324d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:43.363912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bb721-ca7a-4cb4-8bd2-c3111b5831e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:43.364523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bb721-ca7a-4cb4-8bd2-c3111b5831e4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:43.364837 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bb721-ca7a-4cb4-8bd2-c3111b5831e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:43.365110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bb721-ca7a-4cb4-8bd2-c3111b5831e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:43.365869 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bb721-ca7a-4cb4-8bd2-c3111b5831e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:43.366422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bb721-ca7a-4cb4-8bd2-c3111b5831e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:43.367856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bb721-ca7a-4cb4-8bd2-c3111b5831e4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:43.380877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 315/625] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:43.382108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bb721-ca7a-4cb4-8bd2-c3111b5831e4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:43.483354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bb721-ca7a-4cb4-8bd2-c3111b5831e4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:43.505000 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f9a37-f7cd-4a92-aebf-43b10fa1324d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:43.518065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479db1c-039f-44cd-9730-e60d27971ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:43.518703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479db1c-039f-44cd-9730-e60d27971ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:43.518977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479db1c-039f-44cd-9730-e60d27971ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:43.519244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479db1c-039f-44cd-9730-e60d27971ed4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:43.530133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d479db1c-039f-44cd-9730-e60d27971ed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d479db1c-039f-44cd-9730-e60d27971ed4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:43.535015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:43.535946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:43.536817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:43.536817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:43.536817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:43.536817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:43.536817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:43.536817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:43.536817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:43.536817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:43.536817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:43.536817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:43.536817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:43.536817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:43.536817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 316/626] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:43.537509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479db1c-039f-44cd-9730-e60d27971ed4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:43.545821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c0c3-9246-4f1e-9890-4e01c2934612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:43.546541 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c0c3-9246-4f1e-9890-4e01c2934612 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:43.546818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c0c3-9246-4f1e-9890-4e01c2934612 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:43.547083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c0c3-9246-4f1e-9890-4e01c2934612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:43.560923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344c0c3-9246-4f1e-9890-4e01c2934612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:43.562510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344c0c3-9246-4f1e-9890-4e01c2934612 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1344c0c3-9246-4f1e-9890-4e01c2934612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:43.579295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:43.585136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:43.586049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:43.586049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:43.586049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:43.586049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 311/627] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:00:43.586049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c0c3-9246-4f1e-9890-4e01c2934612 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:43.592966 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:43.625034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:43.651721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479db1c-039f-44cd-9730-e60d27971ed4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:43.687873 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeef229-b2f0-4c2f-b4ad-a260e58373ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:43.688571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeef229-b2f0-4c2f-b4ad-a260e58373ba None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:43.688876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeef229-b2f0-4c2f-b4ad-a260e58373ba None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:43.689147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeef229-b2f0-4c2f-b4ad-a260e58373ba None admin] PATH_INFO: `/v3/projects/05af249ba0a047fe83aefb078e203e53/users/061c584de8264868af8782402a98a72e/roles/b3f49281e89f4fa1a2d119e2fb4922e5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:43.700303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c0c3-9246-4f1e-9890-4e01c2934612 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:43.703846 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeef229-b2f0-4c2f-b4ad-a260e58373ba None admin] RBAC: Authorizing `identity:create_grant(project_id=05af249ba0a047fe83aefb078e203e53, user_id=061c584de8264868af8782402a98a72e, role_id=b3f49281e89f4fa1a2d119e2fb4922e5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:43.705349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeef229-b2f0-4c2f-b4ad-a260e58373ba None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:43.729509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 317/628] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:00:43 2022] PUT /identity/v3/projects/05af249ba0a047fe83aefb078e203e53/users/061c584de8264868af8782402a98a72e/roles/b3f49281e89f4fa1a2d119e2fb4922e5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:00:43.729509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeef229-b2f0-4c2f-b4ad-a260e58373ba None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:43.760636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae2cc5-dbd3-4233-a187-6f2c826a592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:43.761860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae2cc5-dbd3-4233-a187-6f2c826a592b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:43.762424 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae2cc5-dbd3-4233-a187-6f2c826a592b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:43.763007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae2cc5-dbd3-4233-a187-6f2c826a592b None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:43.763959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae2cc5-dbd3-4233-a187-6f2c826a592b None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:43.764508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae2cc5-dbd3-4233-a187-6f2c826a592b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:43.767610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae2cc5-dbd3-4233-a187-6f2c826a592b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:43.788191 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 312/629] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:43.791709 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fbfcc-4a25-466d-94bf-24924979f7c7 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:43.792399 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fbfcc-4a25-466d-94bf-24924979f7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:43.793002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fbfcc-4a25-466d-94bf-24924979f7c7 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:43.796610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 313/630] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:43.798293 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9fbfcc-4a25-466d-94bf-24924979f7c7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:43.852816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:43.865780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeef229-b2f0-4c2f-b4ad-a260e58373ba None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:43.919741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006bc60-dee5-4021-be07-e7ac2e52f4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:43.920963 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006bc60-dee5-4021-be07-e7ac2e52f4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:43.921443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006bc60-dee5-4021-be07-e7ac2e52f4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:43.922164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006bc60-dee5-4021-be07-e7ac2e52f4c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:43.923590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006bc60-dee5-4021-be07-e7ac2e52f4c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:43.924001 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006bc60-dee5-4021-be07-e7ac2e52f4c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:43.925785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006bc60-dee5-4021-be07-e7ac2e52f4c3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:43.934386 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:43.939685 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 318/631] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:43.941181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d0014-eaf7-4502-ad83-7f354d8006eb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:43.941588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d0014-eaf7-4502-ad83-7f354d8006eb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:43.941959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d0014-eaf7-4502-ad83-7f354d8006eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:43.949217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9fbfcc-4a25-466d-94bf-24924979f7c7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:43.962503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9c4d0014-eaf7-4502-ad83-7f354d8006eb None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:43.963906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9c4d0014-eaf7-4502-ad83-7f354d8006eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:43.988344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10291ba8-728a-4830-9bb9-4e256b0cc183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:43.991295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10291ba8-728a-4830-9bb9-4e256b0cc183 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:43.991295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10291ba8-728a-4830-9bb9-4e256b0cc183 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:43.991295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10291ba8-728a-4830-9bb9-4e256b0cc183 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:44.003356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10291ba8-728a-4830-9bb9-4e256b0cc183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:44.005571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10291ba8-728a-4830-9bb9-4e256b0cc183 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-10291ba8-728a-4830-9bb9-4e256b0cc183 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:44.022471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:44.023650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:44.024847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:44.024847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:44.024847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:44.028838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 314/632] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:44.028838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10291ba8-728a-4830-9bb9-4e256b0cc183 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:44.062448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 319/633] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:44.063730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d0014-eaf7-4502-ad83-7f354d8006eb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:44.170898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:44.195262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d0014-eaf7-4502-ad83-7f354d8006eb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:44.231286 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10291ba8-728a-4830-9bb9-4e256b0cc183 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:44.239374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e210ef-6578-466b-8960-30963bcbca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:44.240611 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e210ef-6578-466b-8960-30963bcbca2f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:44.241061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e210ef-6578-466b-8960-30963bcbca2f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:44.241475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e210ef-6578-466b-8960-30963bcbca2f None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:44.242644 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e210ef-6578-466b-8960-30963bcbca2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:44.243117 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e210ef-6578-466b-8960-30963bcbca2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:44.245350 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e210ef-6578-466b-8960-30963bcbca2f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:44.265629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 320/634] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:44.266805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042329e0-5a1b-4809-bf5d-4093408f9251 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:44.267057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042329e0-5a1b-4809-bf5d-4093408f9251 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:44.267329 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042329e0-5a1b-4809-bf5d-4093408f9251 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:44.268967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 321/635] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:44.269689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042329e0-5a1b-4809-bf5d-4093408f9251 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:44.281256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c600f-afb7-4249-b5e0-2cd3dd742fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:44.283004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c600f-afb7-4249-b5e0-2cd3dd742fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:44.283432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c600f-afb7-4249-b5e0-2cd3dd742fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:44.283863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c600f-afb7-4249-b5e0-2cd3dd742fc9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:44.296634 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c600f-afb7-4249-b5e0-2cd3dd742fc9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:44.298880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c600f-afb7-4249-b5e0-2cd3dd742fc9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a76c600f-afb7-4249-b5e0-2cd3dd742fc9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:44.315307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:44.316535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:44.317738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:44.317738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:44.317738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:44.317738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:44.317738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:44.317738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:44.317738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:44.317738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:44.317738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:00:44.317738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:44.325433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 315/636] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Sep 6 21:00:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:44.325433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66303f-d53d-46be-991c-dcea5795eec6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:44.325433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66303f-d53d-46be-991c-dcea5795eec6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:44.325433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66303f-d53d-46be-991c-dcea5795eec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:44.332529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:44.347342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-df66303f-d53d-46be-991c-dcea5795eec6 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:44.348970 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-df66303f-d53d-46be-991c-dcea5795eec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:44.387546 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042329e0-5a1b-4809-bf5d-4093408f9251 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:44.424790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45ec85-1667-4726-94fe-c3642e178a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:44.425438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45ec85-1667-4726-94fe-c3642e178a51 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:44.425721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45ec85-1667-4726-94fe-c3642e178a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:44.425995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45ec85-1667-4726-94fe-c3642e178a51 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:44.427342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 316/637] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:44.428329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66303f-d53d-46be-991c-dcea5795eec6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:44.437902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45ec85-1667-4726-94fe-c3642e178a51 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:44.439963 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45ec85-1667-4726-94fe-c3642e178a51 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ce45ec85-1667-4726-94fe-c3642e178a51 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:44.457378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:44.458487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:44.459430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:44.459430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:44.459430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:44.459430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:44.459430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:44.459430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:44.459430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:44.459430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:44.459430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 21:00:44.459430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:44.461008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 322/638] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:44.462200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45ec85-1667-4726-94fe-c3642e178a51 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:44.468470 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:44.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66303f-d53d-46be-991c-dcea5795eec6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:44.577971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:44.586316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9ea08-8f9a-4ed9-87ac-9ea0ec9301a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:44.587196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9ea08-8f9a-4ed9-87ac-9ea0ec9301a5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:44.587301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9ea08-8f9a-4ed9-87ac-9ea0ec9301a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:44.587499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9ea08-8f9a-4ed9-87ac-9ea0ec9301a5 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:44.590548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9ea08-8f9a-4ed9-87ac-9ea0ec9301a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:44.590873 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9ea08-8f9a-4ed9-87ac-9ea0ec9301a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:44.592742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9ea08-8f9a-4ed9-87ac-9ea0ec9301a5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:44.610633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45ec85-1667-4726-94fe-c3642e178a51 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:44.617139 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 317/639] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Sep 6 21:00:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:44.618320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9ea08-8f9a-4ed9-87ac-9ea0ec9301a5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:44.651607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb8833-b585-4286-88b2-212b2593f840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:44.652744 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb8833-b585-4286-88b2-212b2593f840 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:44.653279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb8833-b585-4286-88b2-212b2593f840 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:44.653739 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb8833-b585-4286-88b2-212b2593f840 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:44.665050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb8833-b585-4286-88b2-212b2593f840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2efb8833-b585-4286-88b2-212b2593f840 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:44.671147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:44.672076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:44.673316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:44.673316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:44.673316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:44.673316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:44.673316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:44.673316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:44.673316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:44.673316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:44.673316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:44.673316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:44.673316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:44.673316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:44.675760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 323/640] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:44.676861 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb8833-b585-4286-88b2-212b2593f840 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:44.744398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9ea08-8f9a-4ed9-87ac-9ea0ec9301a5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:44.745223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:44.788753 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0783ed-3353-44fc-92dd-95d91aa66c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:44.789376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0783ed-3353-44fc-92dd-95d91aa66c28 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:44.789654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0783ed-3353-44fc-92dd-95d91aa66c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:44.789932 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0783ed-3353-44fc-92dd-95d91aa66c28 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:44.790704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0783ed-3353-44fc-92dd-95d91aa66c28 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:44.791009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0783ed-3353-44fc-92dd-95d91aa66c28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:44.791933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb8833-b585-4286-88b2-212b2593f840 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:44.792472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0783ed-3353-44fc-92dd-95d91aa66c28 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:44.814937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 318/641] 10.222.0.79 () {56 vars in 1137 bytes} [Tue Sep 6 21:00:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:44.820902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0783ed-3353-44fc-92dd-95d91aa66c28 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:44.829500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b163b-7ebb-4137-994d-e7846e74283d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:44.830113 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b163b-7ebb-4137-994d-e7846e74283d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:44.830389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b163b-7ebb-4137-994d-e7846e74283d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:44.830657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b163b-7ebb-4137-994d-e7846e74283d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:44.871847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:44.905749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b163b-7ebb-4137-994d-e7846e74283d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:44.907233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b163b-7ebb-4137-994d-e7846e74283d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:44.924241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0783ed-3353-44fc-92dd-95d91aa66c28 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:44.962627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170f505-ee4a-43a9-8e90-4daffcef3402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:44.963249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170f505-ee4a-43a9-8e90-4daffcef3402 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:44.963527 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170f505-ee4a-43a9-8e90-4daffcef3402 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:44.963793 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170f505-ee4a-43a9-8e90-4daffcef3402 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:44.964458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170f505-ee4a-43a9-8e90-4daffcef3402 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:44.964805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170f505-ee4a-43a9-8e90-4daffcef3402 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:44.966252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170f505-ee4a-43a9-8e90-4daffcef3402 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:44.971692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:44.975879 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 319/642] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:44.977264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170f505-ee4a-43a9-8e90-4daffcef3402 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:45.027510 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 324/643] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:45.028361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b163b-7ebb-4137-994d-e7846e74283d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:45.095304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170f505-ee4a-43a9-8e90-4daffcef3402 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:45.107045 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:45.132218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54f75e-8324-480a-8759-7b9862e6eb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:45.132890 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54f75e-8324-480a-8759-7b9862e6eb06 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:45.133592 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54f75e-8324-480a-8759-7b9862e6eb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:45.133871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54f75e-8324-480a-8759-7b9862e6eb06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:45.145503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54f75e-8324-480a-8759-7b9862e6eb06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:45.148954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b163b-7ebb-4137-994d-e7846e74283d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6a54f75e-8324-480a-8759-7b9862e6eb06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:45.150924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:45.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 320/644] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:45.153827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54f75e-8324-480a-8759-7b9862e6eb06 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:45.199745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595047ba-09a0-4957-8f82-e3dc274d23b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:45.200368 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595047ba-09a0-4957-8f82-e3dc274d23b2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:45.200686 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595047ba-09a0-4957-8f82-e3dc274d23b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:45.200935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595047ba-09a0-4957-8f82-e3dc274d23b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:45.234676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:45.266237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54f75e-8324-480a-8759-7b9862e6eb06 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:45.278298 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595047ba-09a0-4957-8f82-e3dc274d23b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:45.279759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595047ba-09a0-4957-8f82-e3dc274d23b2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:45.306023 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a38d77-8a1c-4226-9b59-e237f2f5c404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:45.306603 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38d77-8a1c-4226-9b59-e237f2f5c404 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:45.306885 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38d77-8a1c-4226-9b59-e237f2f5c404 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:45.307119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38d77-8a1c-4226-9b59-e237f2f5c404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:45.318690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a38d77-8a1c-4226-9b59-e237f2f5c404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:45.320147 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a38d77-8a1c-4226-9b59-e237f2f5c404 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a6a38d77-8a1c-4226-9b59-e237f2f5c404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:45.335256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:45.336388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:45.337792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:45.337792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:45.337792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:45.341206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 321/645] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:45.341821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a38d77-8a1c-4226-9b59-e237f2f5c404 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:45.368116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:45.390596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 325/646] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:45.391470 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595047ba-09a0-4957-8f82-e3dc274d23b2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:45.452875 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a38d77-8a1c-4226-9b59-e237f2f5c404 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:45.458269 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:45.487630 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1839497-850b-4d8d-8886-c3a532ab62b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:45.488260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1839497-850b-4d8d-8886-c3a532ab62b3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:45.488532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1839497-850b-4d8d-8886-c3a532ab62b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:45.488841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1839497-850b-4d8d-8886-c3a532ab62b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:45.499652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1839497-850b-4d8d-8886-c3a532ab62b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:45.501268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595047ba-09a0-4957-8f82-e3dc274d23b2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:45.501378 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1839497-850b-4d8d-8886-c3a532ab62b3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e1839497-850b-4d8d-8886-c3a532ab62b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:45.517508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:45.518718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:45.519845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:45.519845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:45.519845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:45.525727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 322/647] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:45.525727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1839497-850b-4d8d-8886-c3a532ab62b3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:45.550214 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df2f00-d74d-4b43-b585-7727013738f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:45.550850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df2f00-d74d-4b43-b585-7727013738f4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:45.551131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df2f00-d74d-4b43-b585-7727013738f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:45.551406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df2f00-d74d-4b43-b585-7727013738f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:45.552084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df2f00-d74d-4b43-b585-7727013738f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:45.552382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df2f00-d74d-4b43-b585-7727013738f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:45.553832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df2f00-d74d-4b43-b585-7727013738f4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:45.568594 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 326/648] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:45.568594 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df2f00-d74d-4b43-b585-7727013738f4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:45.621509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:45.648959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1839497-850b-4d8d-8886-c3a532ab62b3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:45.670469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df2f00-d74d-4b43-b585-7727013738f4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:45.683993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d237406-d229-45d3-90e5-4a03529f0280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:45.684749 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d237406-d229-45d3-90e5-4a03529f0280 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:45.685037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d237406-d229-45d3-90e5-4a03529f0280 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:45.685693 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d237406-d229-45d3-90e5-4a03529f0280 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:45.686444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d237406-d229-45d3-90e5-4a03529f0280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:45.686856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d237406-d229-45d3-90e5-4a03529f0280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:45.688341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d237406-d229-45d3-90e5-4a03529f0280 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:45.693558 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:45.706447 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 323/649] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:45.706447 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d237406-d229-45d3-90e5-4a03529f0280 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:45.709252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4444e19-ad1b-4d1a-99cc-75d9c8178b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:45.709891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4444e19-ad1b-4d1a-99cc-75d9c8178b89 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:45.710175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4444e19-ad1b-4d1a-99cc-75d9c8178b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:45.710445 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4444e19-ad1b-4d1a-99cc-75d9c8178b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:45.722902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4444e19-ad1b-4d1a-99cc-75d9c8178b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:45.724374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4444e19-ad1b-4d1a-99cc-75d9c8178b89 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c4444e19-ad1b-4d1a-99cc-75d9c8178b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:45.738926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:45.739931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:45.740851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:45.740851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:45.740851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:45.740851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 327/650] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:45.740851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4444e19-ad1b-4d1a-99cc-75d9c8178b89 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:45.819711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d237406-d229-45d3-90e5-4a03529f0280 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:45.852620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4444e19-ad1b-4d1a-99cc-75d9c8178b89 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:45.858113 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989d020-d3c6-485a-90b8-7fb0e84bb4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:45.858747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989d020-d3c6-485a-90b8-7fb0e84bb4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:45.859025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989d020-d3c6-485a-90b8-7fb0e84bb4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:45.859295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989d020-d3c6-485a-90b8-7fb0e84bb4e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:45.859958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989d020-d3c6-485a-90b8-7fb0e84bb4e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:45.860264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989d020-d3c6-485a-90b8-7fb0e84bb4e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:45.861755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989d020-d3c6-485a-90b8-7fb0e84bb4e0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:45.863498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:45.876364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 324/651] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:45.877212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989d020-d3c6-485a-90b8-7fb0e84bb4e0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:45.895774 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c54f7-8301-4772-9737-f21425f89687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:45.896528 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c54f7-8301-4772-9737-f21425f89687 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:45.896900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c54f7-8301-4772-9737-f21425f89687 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:45.897232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c54f7-8301-4772-9737-f21425f89687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:45.909064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c54f7-8301-4772-9737-f21425f89687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:45.910602 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c54f7-8301-4772-9737-f21425f89687 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-976c54f7-8301-4772-9737-f21425f89687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:45.924996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:45.926164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:45.927242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:45.927242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:45.927242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:45.932447 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 328/652] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:45.932447 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c54f7-8301-4772-9737-f21425f89687 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:45.959943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:46.005934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989d020-d3c6-485a-90b8-7fb0e84bb4e0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:46.032327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c54f7-8301-4772-9737-f21425f89687 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:46.039411 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a9553-57c8-4c82-8a05-3a4662d8839f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:46.040034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a9553-57c8-4c82-8a05-3a4662d8839f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:46.040285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a9553-57c8-4c82-8a05-3a4662d8839f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:46.040615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a9553-57c8-4c82-8a05-3a4662d8839f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:46.048449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a9553-57c8-4c82-8a05-3a4662d8839f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:46.049929 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a9553-57c8-4c82-8a05-3a4662d8839f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9c9a9553-57c8-4c82-8a05-3a4662d8839f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:46.062476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:46.063585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:46.064684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:46.064684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:46.064684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:46.064684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:46.064684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:46.064684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:46.064684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:46.064684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:46.064684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 21:00:46.064684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:46.064684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 325/653] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:46.064684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a9553-57c8-4c82-8a05-3a4662d8839f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:46.071669 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3f8cf-fc37-4f2e-b105-805b04314248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:46.072059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3f8cf-fc37-4f2e-b105-805b04314248 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:46.072356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3f8cf-fc37-4f2e-b105-805b04314248 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:46.072626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3f8cf-fc37-4f2e-b105-805b04314248 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:46.073289 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3f8cf-fc37-4f2e-b105-805b04314248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:46.073583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3f8cf-fc37-4f2e-b105-805b04314248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:46.075014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3f8cf-fc37-4f2e-b105-805b04314248 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:46.087923 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 329/654] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:46.088773 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3f8cf-fc37-4f2e-b105-805b04314248 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:46.125390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:46.172188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a9553-57c8-4c82-8a05-3a4662d8839f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:46.178752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:46.191732 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3f8cf-fc37-4f2e-b105-805b04314248 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:46.207137 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eea36-4b68-41a8-b7be-456f9fd94aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:46.207772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eea36-4b68-41a8-b7be-456f9fd94aaa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:46.208033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eea36-4b68-41a8-b7be-456f9fd94aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:46.208318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eea36-4b68-41a8-b7be-456f9fd94aaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:46.214810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907eea36-4b68-41a8-b7be-456f9fd94aaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:46.216293 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907eea36-4b68-41a8-b7be-456f9fd94aaa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:46.225438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d560a-33df-4f76-9d24-c29ac456e92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:46.226046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d560a-33df-4f76-9d24-c29ac456e92c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:46.226335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d560a-33df-4f76-9d24-c29ac456e92c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:46.226598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d560a-33df-4f76-9d24-c29ac456e92c None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-907eea36-4b68-41a8-b7be-456f9fd94aaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:46.227200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:46.233403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:46.234517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:46.234517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:46.234517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:46.234517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:46.234517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:46.234517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:46.234517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:46.234517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:46.235020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d560a-33df-4f76-9d24-c29ac456e92c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:46.235020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d560a-33df-4f76-9d24-c29ac456e92c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:46.235180 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 326/655] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:46.235259 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d560a-33df-4f76-9d24-c29ac456e92c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:46.235348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97683bb-6437-4968-b136-6c350ca55133 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:46.235348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97683bb-6437-4968-b136-6c350ca55133 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:46.235348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97683bb-6437-4968-b136-6c350ca55133 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:46.235348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 327/656] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:46.236280 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97683bb-6437-4968-b136-6c350ca55133 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:46.248137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 330/657] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:46.248137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d560a-33df-4f76-9d24-c29ac456e92c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:46.340796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97683bb-6437-4968-b136-6c350ca55133 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:46.344207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d560a-33df-4f76-9d24-c29ac456e92c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:46.354288 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:46.375277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f97a0-3749-48d1-8d68-20989b700f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:46.375896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f97a0-3749-48d1-8d68-20989b700f35 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:46.376177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f97a0-3749-48d1-8d68-20989b700f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:46.376491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f97a0-3749-48d1-8d68-20989b700f35 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:46.377222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f97a0-3749-48d1-8d68-20989b700f35 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:46.377525 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f97a0-3749-48d1-8d68-20989b700f35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:46.378987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f97a0-3749-48d1-8d68-20989b700f35 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:46.380913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffca1a-8cd9-4eed-bd8f-8e15ac8f6a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:46.381552 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffca1a-8cd9-4eed-bd8f-8e15ac8f6a3d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:46.381831 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffca1a-8cd9-4eed-bd8f-8e15ac8f6a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:46.382101 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffca1a-8cd9-4eed-bd8f-8e15ac8f6a3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:46.388879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffca1a-8cd9-4eed-bd8f-8e15ac8f6a3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:46.390323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffca1a-8cd9-4eed-bd8f-8e15ac8f6a3d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:46.395912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 328/658] 10.222.0.79 () {56 vars in 1137 bytes} [Tue Sep 6 21:00:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:46.396817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f97a0-3749-48d1-8d68-20989b700f35 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-64ffca1a-8cd9-4eed-bd8f-8e15ac8f6a3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:46.402833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:46.404282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:46.405299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:46.405299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:46.405299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:46.405299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:46.405299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:46.405299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:46.405299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:46.405299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:46.408644 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 331/659] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:46.409729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e76040-c0c0-4a67-9184-c32af9f7b71e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:46.410009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e76040-c0c0-4a67-9184-c32af9f7b71e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:46.410277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e76040-c0c0-4a67-9184-c32af9f7b71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:46.428402 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e7e76040-c0c0-4a67-9184-c32af9f7b71e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:46.429580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e7e76040-c0c0-4a67-9184-c32af9f7b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:46.431650 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:46.497129 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f97a0-3749-48d1-8d68-20989b700f35 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:46.498611 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 332/660] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:46.499447 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e76040-c0c0-4a67-9184-c32af9f7b71e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:46.533985 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79894fa2-29dc-44e2-aaa3-91831ab18e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:46.534878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79894fa2-29dc-44e2-aaa3-91831ab18e82 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:46.535300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79894fa2-29dc-44e2-aaa3-91831ab18e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:46.535727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79894fa2-29dc-44e2-aaa3-91831ab18e82 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:46.536557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79894fa2-29dc-44e2-aaa3-91831ab18e82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:46.537024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79894fa2-29dc-44e2-aaa3-91831ab18e82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:46.538620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79894fa2-29dc-44e2-aaa3-91831ab18e82 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:46.556231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 329/661] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:46.557410 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79894fa2-29dc-44e2-aaa3-91831ab18e82 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:46.593517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:46.606565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e76040-c0c0-4a67-9184-c32af9f7b71e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:46.642305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3f9a2-a849-4f86-b00c-a3491530f735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:46.642946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3f9a2-a849-4f86-b00c-a3491530f735 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:46.643527 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3f9a2-a849-4f86-b00c-a3491530f735 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:46.643633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3f9a2-a849-4f86-b00c-a3491530f735 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:46.652754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3f9a2-a849-4f86-b00c-a3491530f735 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:46.655073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3f9a2-a849-4f86-b00c-a3491530f735 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:46.657362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:46.667058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79894fa2-29dc-44e2-aaa3-91831ab18e82 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4ec3f9a2-a849-4f86-b00c-a3491530f735 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:46.669433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:46.670327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:46.671314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:46.671314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:46.671314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:46.671314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:46.671314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:46.671314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:46.671314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:46.671314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:46.671314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 21:00:46.671314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:46.671314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 333/662] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:46.671314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3f9a2-a849-4f86-b00c-a3491530f735 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:46.706779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec79b83-f26f-47ec-a720-ab0983fa8239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:46.707309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec79b83-f26f-47ec-a720-ab0983fa8239 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:46.707584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec79b83-f26f-47ec-a720-ab0983fa8239 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:46.707848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec79b83-f26f-47ec-a720-ab0983fa8239 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:46.781850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3f9a2-a849-4f86-b00c-a3491530f735 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:46.789727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec79b83-f26f-47ec-a720-ab0983fa8239 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:46.791234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec79b83-f26f-47ec-a720-ab0983fa8239 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:46.818033 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea15cc8-13f5-4ae9-a08d-53c6206ab306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:46.818647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea15cc8-13f5-4ae9-a08d-53c6206ab306 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:46.818924 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea15cc8-13f5-4ae9-a08d-53c6206ab306 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:46.819189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea15cc8-13f5-4ae9-a08d-53c6206ab306 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:46.819877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea15cc8-13f5-4ae9-a08d-53c6206ab306 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:46.820180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea15cc8-13f5-4ae9-a08d-53c6206ab306 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:46.821211 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:46.821728 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea15cc8-13f5-4ae9-a08d-53c6206ab306 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:46.834388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 334/663] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:46.835196 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea15cc8-13f5-4ae9-a08d-53c6206ab306 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:46.892724 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 330/664] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:46.893765 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec79b83-f26f-47ec-a720-ab0983fa8239 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:46.894715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:46.937344 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea15cc8-13f5-4ae9-a08d-53c6206ab306 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:46.978399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570f628-7fa5-44d5-adab-94d63b72856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:46.979016 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570f628-7fa5-44d5-adab-94d63b72856e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:46.979356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570f628-7fa5-44d5-adab-94d63b72856e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:46.979693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570f628-7fa5-44d5-adab-94d63b72856e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:46.980473 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570f628-7fa5-44d5-adab-94d63b72856e None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=12d51454d20c46d7a5e6d963a2f987cc) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:46.980806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570f628-7fa5-44d5-adab-94d63b72856e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:46.982367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570f628-7fa5-44d5-adab-94d63b72856e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:46.988167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 335/665] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:46 2022] GET /identity/v3/role_assignments?user.id=12d51454d20c46d7a5e6d963a2f987cc&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:46.989073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570f628-7fa5-44d5-adab-94d63b72856e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:47.008818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec79b83-f26f-47ec-a720-ab0983fa8239 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:47.043596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:47.050812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53154564-b111-47d8-8c1e-d85c303e6bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:47.051477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53154564-b111-47d8-8c1e-d85c303e6bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:47.051740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53154564-b111-47d8-8c1e-d85c303e6bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:47.051983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53154564-b111-47d8-8c1e-d85c303e6bc1 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:47.052767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53154564-b111-47d8-8c1e-d85c303e6bc1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:47.053102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53154564-b111-47d8-8c1e-d85c303e6bc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:47.054607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53154564-b111-47d8-8c1e-d85c303e6bc1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:47.082818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 331/666] 10.222.0.79 () {56 vars in 1139 bytes} [Tue Sep 6 21:00:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:47.082818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53154564-b111-47d8-8c1e-d85c303e6bc1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:47.111724 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570f628-7fa5-44d5-adab-94d63b72856e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:47.139198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:47.156315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a678091-b3f8-4f1b-ac8d-6c6f3471a8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:47.157002 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a678091-b3f8-4f1b-ac8d-6c6f3471a8cf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:47.157425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a678091-b3f8-4f1b-ac8d-6c6f3471a8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:47.157722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a678091-b3f8-4f1b-ac8d-6c6f3471a8cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:47.170643 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a678091-b3f8-4f1b-ac8d-6c6f3471a8cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4a678091-b3f8-4f1b-ac8d-6c6f3471a8cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:47.176252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:47.177263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:47.178098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:47.178098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:47.178098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:47.178098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:47.178098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:47.178098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:47.178098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:47.178098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:47.178098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:47.178098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:47.178098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:47.178098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:47.178717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 336/667] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:47.179329 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a678091-b3f8-4f1b-ac8d-6c6f3471a8cf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:47.200521 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53154564-b111-47d8-8c1e-d85c303e6bc1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:47.240091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d0176-3c30-4107-9e67-9b2a0ce982c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:47.241026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d0176-3c30-4107-9e67-9b2a0ce982c5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:47.241452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d0176-3c30-4107-9e67-9b2a0ce982c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:47.241727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d0176-3c30-4107-9e67-9b2a0ce982c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:47.242463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d0176-3c30-4107-9e67-9b2a0ce982c5 None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=a76937ab52e445c59b9eb9638fd4a753) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:47.242830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d0176-3c30-4107-9e67-9b2a0ce982c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:47.244422 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d0176-3c30-4107-9e67-9b2a0ce982c5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:47.251312 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 332/668] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:47 2022] GET /identity/v3/role_assignments?user.id=a76937ab52e445c59b9eb9638fd4a753&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:47.252131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d0176-3c30-4107-9e67-9b2a0ce982c5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:47.300308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a678091-b3f8-4f1b-ac8d-6c6f3471a8cf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:47.335603 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:47.346730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bad09-dbb5-4139-bcb8-24d0668c37b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:47.347895 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bad09-dbb5-4139-bcb8-24d0668c37b3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:47.348324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bad09-dbb5-4139-bcb8-24d0668c37b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:47.348759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bad09-dbb5-4139-bcb8-24d0668c37b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:47.364628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bad09-dbb5-4139-bcb8-24d0668c37b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:47.367322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bad09-dbb5-4139-bcb8-24d0668c37b3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-5b9bad09-dbb5-4139-bcb8-24d0668c37b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:47.389541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:47.390700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:47.391759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:47.391759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:47.391759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:47.394564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 337/669] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:47.396481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bad09-dbb5-4139-bcb8-24d0668c37b3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:47.399193 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d0176-3c30-4107-9e67-9b2a0ce982c5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:47.445288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:47.445849 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e804ca6-6448-49c6-a057-35b69a2abb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:47.446289 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e804ca6-6448-49c6-a057-35b69a2abb63 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:47.447265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e804ca6-6448-49c6-a057-35b69a2abb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:47.447642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e804ca6-6448-49c6-a057-35b69a2abb63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:47.528730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bad09-dbb5-4139-bcb8-24d0668c37b3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:47.544796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e804ca6-6448-49c6-a057-35b69a2abb63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:47.546316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e804ca6-6448-49c6-a057-35b69a2abb63 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:47.578528 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eae418-766a-47bb-9758-5718ac08d020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:47.579139 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eae418-766a-47bb-9758-5718ac08d020 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:47.579412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eae418-766a-47bb-9758-5718ac08d020 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:47.579680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eae418-766a-47bb-9758-5718ac08d020 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:47.580348 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eae418-766a-47bb-9758-5718ac08d020 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:47.580647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eae418-766a-47bb-9758-5718ac08d020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:47.586700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eae418-766a-47bb-9758-5718ac08d020 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:47.600680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 338/670] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:47.600680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eae418-766a-47bb-9758-5718ac08d020 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:47.623448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:47.678204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 333/671] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:47.679375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e804ca6-6448-49c6-a057-35b69a2abb63 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:47.704590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:47.711891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eae418-766a-47bb-9758-5718ac08d020 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:47.752141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c19dfe-02d0-4ecd-9e0a-26be428e35a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:47.752939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c19dfe-02d0-4ecd-9e0a-26be428e35a9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:47.753312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c19dfe-02d0-4ecd-9e0a-26be428e35a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:47.753755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c19dfe-02d0-4ecd-9e0a-26be428e35a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:47.754499 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c19dfe-02d0-4ecd-9e0a-26be428e35a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:47.754992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c19dfe-02d0-4ecd-9e0a-26be428e35a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:47.756563 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c19dfe-02d0-4ecd-9e0a-26be428e35a9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:47.770159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 339/672] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:47.771266 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c19dfe-02d0-4ecd-9e0a-26be428e35a9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:47.789889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e804ca6-6448-49c6-a057-35b69a2abb63 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:47.823282 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444297f7-6ec5-4d22-a63d-306a64f2a856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:47.823971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444297f7-6ec5-4d22-a63d-306a64f2a856 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:47.824249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444297f7-6ec5-4d22-a63d-306a64f2a856 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:47.824514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444297f7-6ec5-4d22-a63d-306a64f2a856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:47.835173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444297f7-6ec5-4d22-a63d-306a64f2a856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:47.836633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444297f7-6ec5-4d22-a63d-306a64f2a856 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-444297f7-6ec5-4d22-a63d-306a64f2a856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:47.851739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:47.852926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:47.854002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:47.854002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:47.854002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:47.857860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 334/673] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:47.858738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444297f7-6ec5-4d22-a63d-306a64f2a856 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:47.869171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:47.881438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c19dfe-02d0-4ecd-9e0a-26be428e35a9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:47.919251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8305c86-3b1b-4720-abbe-b95ff61bc350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:47.920325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8305c86-3b1b-4720-abbe-b95ff61bc350 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:47.920833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8305c86-3b1b-4720-abbe-b95ff61bc350 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:47.921326 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8305c86-3b1b-4720-abbe-b95ff61bc350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:47.937136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8305c86-3b1b-4720-abbe-b95ff61bc350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:47.939434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8305c86-3b1b-4720-abbe-b95ff61bc350 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d8305c86-3b1b-4720-abbe-b95ff61bc350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:47.961009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:47.963054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:47.964148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:47.964148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:47.964148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:47.964997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 340/674] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:47.966647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8305c86-3b1b-4720-abbe-b95ff61bc350 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:47.979206 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:48.004732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444297f7-6ec5-4d22-a63d-306a64f2a856 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:48.051656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67da91-147c-4d04-9915-3b67266e7867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:48.053033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67da91-147c-4d04-9915-3b67266e7867 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:48.053732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67da91-147c-4d04-9915-3b67266e7867 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:48.054178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67da91-147c-4d04-9915-3b67266e7867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:48.064559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67da91-147c-4d04-9915-3b67266e7867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:48.066924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67da91-147c-4d04-9915-3b67266e7867 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:48.078595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8305c86-3b1b-4720-abbe-b95ff61bc350 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5f67da91-147c-4d04-9915-3b67266e7867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:48.081012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:48.082069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:48.083173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:48.083173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:48.083173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:48.083173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:48.083173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:48.083173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:48.083173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:48.083173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:48.084800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 335/675] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:48.085767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67da91-147c-4d04-9915-3b67266e7867 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:48.128975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55973d5a-43ee-4099-ab15-feac271b400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:48.130389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55973d5a-43ee-4099-ab15-feac271b400a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:48.130816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55973d5a-43ee-4099-ab15-feac271b400a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:48.131272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55973d5a-43ee-4099-ab15-feac271b400a None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:48.132626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55973d5a-43ee-4099-ab15-feac271b400a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:48.133213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55973d5a-43ee-4099-ab15-feac271b400a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:48.135787 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55973d5a-43ee-4099-ab15-feac271b400a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:48.158678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 341/676] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:48.158678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55973d5a-43ee-4099-ab15-feac271b400a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:48.163286 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:48.200800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67da91-147c-4d04-9915-3b67266e7867 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:48.235497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d0138-406c-41a2-b271-f266425d655a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:48.236265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d0138-406c-41a2-b271-f266425d655a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:48.236763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d0138-406c-41a2-b271-f266425d655a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:48.237135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d0138-406c-41a2-b271-f266425d655a None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:48.237912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d0138-406c-41a2-b271-f266425d655a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:48.238318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d0138-406c-41a2-b271-f266425d655a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:48.239135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:48.240323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d0138-406c-41a2-b271-f266425d655a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:48.255920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 336/677] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:48.257124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d0138-406c-41a2-b271-f266425d655a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:48.282316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55973d5a-43ee-4099-ab15-feac271b400a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:48.325624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71550ec-3bdc-4e35-9d8d-aa888599842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:48.326593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71550ec-3bdc-4e35-9d8d-aa888599842d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:48.327016 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71550ec-3bdc-4e35-9d8d-aa888599842d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:48.327486 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71550ec-3bdc-4e35-9d8d-aa888599842d None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:48.328343 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71550ec-3bdc-4e35-9d8d-aa888599842d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:48.328894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71550ec-3bdc-4e35-9d8d-aa888599842d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:48.330683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71550ec-3bdc-4e35-9d8d-aa888599842d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:48.344320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 342/678] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:48.345626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71550ec-3bdc-4e35-9d8d-aa888599842d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:48.369611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d0138-406c-41a2-b271-f266425d655a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:48.397572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:48.407176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82324068-926f-4389-8db6-cea0d47b3adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:48.407976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82324068-926f-4389-8db6-cea0d47b3adf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:48.408356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82324068-926f-4389-8db6-cea0d47b3adf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:48.408766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82324068-926f-4389-8db6-cea0d47b3adf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:48.416784 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82324068-926f-4389-8db6-cea0d47b3adf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:48.418353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82324068-926f-4389-8db6-cea0d47b3adf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-82324068-926f-4389-8db6-cea0d47b3adf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:48.430986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:48.432138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:48.433361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:48.433361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:48.433361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:48.433361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:48.433361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:48.433361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:48.433361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:48.433361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:48.433361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 21:00:48.433361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:48.440164 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 337/679] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:00:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:48.440164 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82324068-926f-4389-8db6-cea0d47b3adf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:48.470660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71550ec-3bdc-4e35-9d8d-aa888599842d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:48.503672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:48.514820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ce5eb-d5f9-4681-844e-a4d0654d9c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:48.515489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ce5eb-d5f9-4681-844e-a4d0654d9c2c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:48.515767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ce5eb-d5f9-4681-844e-a4d0654d9c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:48.516048 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ce5eb-d5f9-4681-844e-a4d0654d9c2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:48.525664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ce5eb-d5f9-4681-844e-a4d0654d9c2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:48.527131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ce5eb-d5f9-4681-844e-a4d0654d9c2c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-324ce5eb-d5f9-4681-844e-a4d0654d9c2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:48.541915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:48.542867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:48.544954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:48.544954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:48.544954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:48.544954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:48.544954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:48.544954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:48.544954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:48.544954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:48.544954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 343/680] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:48.544954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ce5eb-d5f9-4681-844e-a4d0654d9c2c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:48.577683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82324068-926f-4389-8db6-cea0d47b3adf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:48.630944 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc18400-871e-445a-90e8-8122be36eb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:48.632224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc18400-871e-445a-90e8-8122be36eb28 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:48.632755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc18400-871e-445a-90e8-8122be36eb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:48.634163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc18400-871e-445a-90e8-8122be36eb28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:48.635166 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc18400-871e-445a-90e8-8122be36eb28 None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=f50ae3b9997d4dc39c3d67ca40bbc566) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:48.635668 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc18400-871e-445a-90e8-8122be36eb28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:48.637586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc18400-871e-445a-90e8-8122be36eb28 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:48.646125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 338/681] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:48 2022] GET /identity/v3/role_assignments?user.id=f50ae3b9997d4dc39c3d67ca40bbc566&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:48.647915 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc18400-871e-445a-90e8-8122be36eb28 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:48.687055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ce5eb-d5f9-4681-844e-a4d0654d9c2c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:48.763796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:48.771362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ea5b3-a6b4-44f1-a1fc-c70650b7e2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:48.772357 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ea5b3-a6b4-44f1-a1fc-c70650b7e2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:48.772961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ea5b3-a6b4-44f1-a1fc-c70650b7e2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:48.774067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ea5b3-a6b4-44f1-a1fc-c70650b7e2a7 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:48.775613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ea5b3-a6b4-44f1-a1fc-c70650b7e2a7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:48.776928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ea5b3-a6b4-44f1-a1fc-c70650b7e2a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:48.778879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ea5b3-a6b4-44f1-a1fc-c70650b7e2a7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:48.804743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 344/682] 10.222.0.79 () {56 vars in 1135 bytes} [Tue Sep 6 21:00:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:48.807545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ea5b3-a6b4-44f1-a1fc-c70650b7e2a7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:48.873195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc18400-871e-445a-90e8-8122be36eb28 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:48.896112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:48.920492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbd190-9dd5-4de2-bafd-9d1450139ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:48.921555 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbd190-9dd5-4de2-bafd-9d1450139ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:48.921981 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbd190-9dd5-4de2-bafd-9d1450139ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:48.922397 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbd190-9dd5-4de2-bafd-9d1450139ab9 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:48.923270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbd190-9dd5-4de2-bafd-9d1450139ab9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:48.923988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbd190-9dd5-4de2-bafd-9d1450139ab9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:48.926219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbd190-9dd5-4de2-bafd-9d1450139ab9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:48.944383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 339/683] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:48.949425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbd190-9dd5-4de2-bafd-9d1450139ab9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:48.964821 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ea5b3-a6b4-44f1-a1fc-c70650b7e2a7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:49.004172 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5670b-92e3-4970-88e8-303fdc97851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:49.005008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5670b-92e3-4970-88e8-303fdc97851c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:49.005389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5670b-92e3-4970-88e8-303fdc97851c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:49.005761 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5670b-92e3-4970-88e8-303fdc97851c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:49.084158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbd190-9dd5-4de2-bafd-9d1450139ab9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:49.090147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5670b-92e3-4970-88e8-303fdc97851c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:49.091679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5670b-92e3-4970-88e8-303fdc97851c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:49.099224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:49.135613 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea2309-f24b-4661-ad98-4c508c011fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:49.136151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2309-f24b-4661-ad98-4c508c011fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:49.136426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2309-f24b-4661-ad98-4c508c011fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:49.136724 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2309-f24b-4661-ad98-4c508c011fa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:49.137536 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea2309-f24b-4661-ad98-4c508c011fa7 None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=d5a45dec40844877a6b5b301fc44a110) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:49.139945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea2309-f24b-4661-ad98-4c508c011fa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:49.141385 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea2309-f24b-4661-ad98-4c508c011fa7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:49.159357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 340/684] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:48 2022] GET /identity/v3/role_assignments?user.id=d5a45dec40844877a6b5b301fc44a110&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:49.166763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:49.245490 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 345/685] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 21:00:49.251879 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea2309-f24b-4661-ad98-4c508c011fa7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:49.385964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea2309-f24b-4661-ad98-4c508c011fa7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:49.434611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7168e-b57c-446d-9ee7-a28dcf9bfe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:49.435593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7168e-b57c-446d-9ee7-a28dcf9bfe6d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:49.435883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7168e-b57c-446d-9ee7-a28dcf9bfe6d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:49.436133 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7168e-b57c-446d-9ee7-a28dcf9bfe6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:49.449325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7168e-b57c-446d-9ee7-a28dcf9bfe6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:49.450867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7168e-b57c-446d-9ee7-a28dcf9bfe6d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-56b7168e-b57c-446d-9ee7-a28dcf9bfe6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:49.466391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:49.467369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:49.468242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:49.468242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:49.468242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:49.475588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 341/686] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:49.477228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5670b-92e3-4970-88e8-303fdc97851c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:49.511467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896cad3-9ee9-4b10-99d7-36d52360b1b1 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:49.512152 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896cad3-9ee9-4b10-99d7-36d52360b1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:49.512676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896cad3-9ee9-4b10-99d7-36d52360b1b1 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:49.515247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 342/687] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:49.530489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4b609-b2b7-4c70-8f29-5dd186a0017c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:49.531125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4b609-b2b7-4c70-8f29-5dd186a0017c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:49.531608 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4b609-b2b7-4c70-8f29-5dd186a0017c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:49.556421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e5f4b609-b2b7-4c70-8f29-5dd186a0017c None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:49.558204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e5f4b609-b2b7-4c70-8f29-5dd186a0017c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:49.591961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:49.623749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5670b-92e3-4970-88e8-303fdc97851c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:49.666725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 343/688] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 21:00:49.668574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f64b6-1c7c-4687-8a83-bd39aa434719 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:49.669038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f64b6-1c7c-4687-8a83-bd39aa434719 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:49.669518 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f64b6-1c7c-4687-8a83-bd39aa434719 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:49.671178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 344/689] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:49.685960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a2cd-ca09-42d9-a584-b2143fc9d598 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:49.686473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a2cd-ca09-42d9-a584-b2143fc9d598 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:49.686891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a2cd-ca09-42d9-a584-b2143fc9d598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:49.694871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593222b-7565-4766-b9c4-50f5aef0f478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:49.695704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593222b-7565-4766-b9c4-50f5aef0f478 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:49.696152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593222b-7565-4766-b9c4-50f5aef0f478 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:49.696639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593222b-7565-4766-b9c4-50f5aef0f478 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:49.697455 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593222b-7565-4766-b9c4-50f5aef0f478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:49.699500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593222b-7565-4766-b9c4-50f5aef0f478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:49.701118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593222b-7565-4766-b9c4-50f5aef0f478 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:49.716068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4a84a2cd-ca09-42d9-a584-b2143fc9d598 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:49.718224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4a84a2cd-ca09-42d9-a584-b2143fc9d598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:49.730904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 346/690] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:49.735701 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593222b-7565-4766-b9c4-50f5aef0f478 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:49.755880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:49.809264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 345/691] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 21:00:49.906095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593222b-7565-4766-b9c4-50f5aef0f478 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:49.955437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f634a13-4855-49d7-943e-6fd0cdadf378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:49.956013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f634a13-4855-49d7-943e-6fd0cdadf378 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:49.956290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f634a13-4855-49d7-943e-6fd0cdadf378 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:49.956559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f634a13-4855-49d7-943e-6fd0cdadf378 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:49.970759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f634a13-4855-49d7-943e-6fd0cdadf378 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:49.972230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f634a13-4855-49d7-943e-6fd0cdadf378 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:49.989569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84a2cd-ca09-42d9-a584-b2143fc9d598 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6f634a13-4855-49d7-943e-6fd0cdadf378 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:49.994150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:49.995064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:49.996130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:49.996130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:49.996130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:49.996130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:49.996130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:49.996130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:49.996130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:49.996130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:49.996130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 347/692] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:00:50.010809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f634a13-4855-49d7-943e-6fd0cdadf378 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:50.132272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:50.139482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:50.155835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84a2cd-ca09-42d9-a584-b2143fc9d598 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:50.191219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f634a13-4855-49d7-943e-6fd0cdadf378 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:50.193116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3d4ac-4744-4360-9a5f-120e22cb3313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:50.193766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d4ac-4744-4360-9a5f-120e22cb3313 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:50.194053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d4ac-4744-4360-9a5f-120e22cb3313 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:50.194323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d4ac-4744-4360-9a5f-120e22cb3313 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:50.205272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3d4ac-4744-4360-9a5f-120e22cb3313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ddb3d4ac-4744-4360-9a5f-120e22cb3313 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:50.210365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:50.211271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:50.212128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:50.212128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:50.212128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:50.212128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:50.212128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:50.212128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:50.212128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:50.212128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:50.212128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:50.212128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:50.212128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:00:50.212128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:50.212128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 346/693] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:00:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:50.213356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3d4ac-4744-4360-9a5f-120e22cb3313 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:50.240974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e7796-89a8-4d64-86f3-78d18739ef95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:50.241590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e7796-89a8-4d64-86f3-78d18739ef95 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:50.241866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e7796-89a8-4d64-86f3-78d18739ef95 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:50.242134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e7796-89a8-4d64-86f3-78d18739ef95 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:50.242910 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e7796-89a8-4d64-86f3-78d18739ef95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:50.243325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e7796-89a8-4d64-86f3-78d18739ef95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:50.245142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e7796-89a8-4d64-86f3-78d18739ef95 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:50.264108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 348/694] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:50.265086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e7796-89a8-4d64-86f3-78d18739ef95 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:50.344711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3d4ac-4744-4360-9a5f-120e22cb3313 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:50.375750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e7796-89a8-4d64-86f3-78d18739ef95 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:50.386201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7411e-4373-4107-b6bf-66f70d33da90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:50.386856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7411e-4373-4107-b6bf-66f70d33da90 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:50.387162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7411e-4373-4107-b6bf-66f70d33da90 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:50.387439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7411e-4373-4107-b6bf-66f70d33da90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:50.398094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7411e-4373-4107-b6bf-66f70d33da90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:50.399743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7411e-4373-4107-b6bf-66f70d33da90 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:50.407648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:50.409311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-71f7411e-4373-4107-b6bf-66f70d33da90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:50.417109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:50.418188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:50.419922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:50.419922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:50.419922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:50.423458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 347/695] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:50.424388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7411e-4373-4107-b6bf-66f70d33da90 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:50.424909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f67372-bb52-40af-a4cb-4ee7352ab630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:50.425884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f67372-bb52-40af-a4cb-4ee7352ab630 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:50.426373 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f67372-bb52-40af-a4cb-4ee7352ab630 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:50.426803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f67372-bb52-40af-a4cb-4ee7352ab630 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:50.427774 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f67372-bb52-40af-a4cb-4ee7352ab630 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:50.428239 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f67372-bb52-40af-a4cb-4ee7352ab630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:50.430180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f67372-bb52-40af-a4cb-4ee7352ab630 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:50.441462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 349/696] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:50.442697 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f67372-bb52-40af-a4cb-4ee7352ab630 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:50.553332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7411e-4373-4107-b6bf-66f70d33da90 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:50.572590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f67372-bb52-40af-a4cb-4ee7352ab630 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:50.602799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525f0a5-db4a-4d37-8193-90a7b6235e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:50.603631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525f0a5-db4a-4d37-8193-90a7b6235e7f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:50.604051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525f0a5-db4a-4d37-8193-90a7b6235e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:50.604462 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525f0a5-db4a-4d37-8193-90a7b6235e7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:50.605796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525f0a5-db4a-4d37-8193-90a7b6235e7f None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=061c584de8264868af8782402a98a72e) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:50.606333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525f0a5-db4a-4d37-8193-90a7b6235e7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:50.608355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525f0a5-db4a-4d37-8193-90a7b6235e7f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:50.616902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 348/697] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:50 2022] GET /identity/v3/role_assignments?user.id=061c584de8264868af8782402a98a72e&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:50.617954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525f0a5-db4a-4d37-8193-90a7b6235e7f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:50.621446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da36a9d-6a0d-4a11-aa12-35c18c210e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:50.623256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da36a9d-6a0d-4a11-aa12-35c18c210e7e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:50.623639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da36a9d-6a0d-4a11-aa12-35c18c210e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:50.624012 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da36a9d-6a0d-4a11-aa12-35c18c210e7e None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:50.624830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da36a9d-6a0d-4a11-aa12-35c18c210e7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:50.625248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da36a9d-6a0d-4a11-aa12-35c18c210e7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:50.626791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da36a9d-6a0d-4a11-aa12-35c18c210e7e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:50.654549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 350/698] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:50.661873 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da36a9d-6a0d-4a11-aa12-35c18c210e7e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:50.757087 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:50.768396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:50.828294 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525f0a5-db4a-4d37-8193-90a7b6235e7f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:50.843347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da36a9d-6a0d-4a11-aa12-35c18c210e7e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:50.885146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6517b11-89eb-47f6-ba1e-06a1642e6d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:50.885580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20964971-e7de-40c6-b6e5-4c814f4b2123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:50.886209 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20964971-e7de-40c6-b6e5-4c814f4b2123 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:50.886489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20964971-e7de-40c6-b6e5-4c814f4b2123 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:50.886764 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20964971-e7de-40c6-b6e5-4c814f4b2123 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:50.887135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6517b11-89eb-47f6-ba1e-06a1642e6d5f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:50.887628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20964971-e7de-40c6-b6e5-4c814f4b2123 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:50.887824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6517b11-89eb-47f6-ba1e-06a1642e6d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:50.888295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6517b11-89eb-47f6-ba1e-06a1642e6d5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:50.890372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20964971-e7de-40c6-b6e5-4c814f4b2123 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:50.904881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6517b11-89eb-47f6-ba1e-06a1642e6d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:50.906726 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6517b11-89eb-47f6-ba1e-06a1642e6d5f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:50.923306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 351/699] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 21:00:50 2022] POST /identity/v3/users => generated 254 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c6517b11-89eb-47f6-ba1e-06a1642e6d5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:50.934325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:50.935719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:50.937113 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:50.937113 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:50.937113 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:50.940606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 349/700] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:50.942119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20964971-e7de-40c6-b6e5-4c814f4b2123 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:51.064260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20964971-e7de-40c6-b6e5-4c814f4b2123 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:51.067704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:51.114877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a37fb7-e42c-4406-9668-2d8fba4f40e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:51.115894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a37fb7-e42c-4406-9668-2d8fba4f40e8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:51.116353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a37fb7-e42c-4406-9668-2d8fba4f40e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:51.116934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a37fb7-e42c-4406-9668-2d8fba4f40e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:51.117801 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a37fb7-e42c-4406-9668-2d8fba4f40e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:51.118358 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a37fb7-e42c-4406-9668-2d8fba4f40e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:51.120036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a37fb7-e42c-4406-9668-2d8fba4f40e8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:51.133571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 352/701] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:51.140139 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6517b11-89eb-47f6-ba1e-06a1642e6d5f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:51.256030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6517b11-89eb-47f6-ba1e-06a1642e6d5f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:51.272811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:51.298388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dd36d-e96a-45f0-9377-6625f1296a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:51.299094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd36d-e96a-45f0-9377-6625f1296a4e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:51.299379 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd36d-e96a-45f0-9377-6625f1296a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:51.299654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd36d-e96a-45f0-9377-6625f1296a4e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:51.308198 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8dd36d-e96a-45f0-9377-6625f1296a4e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:51.309819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8dd36d-e96a-45f0-9377-6625f1296a4e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ed8dd36d-e96a-45f0-9377-6625f1296a4e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:51.322314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:51.323300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:51.324305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:51.324305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:51.324305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:51.324305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:51.324305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:51.324305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:51.324305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:51.324305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:51.324305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:00:51.324305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:51.324305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 350/702] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Sep 6 21:00:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:51.331238 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a37fb7-e42c-4406-9668-2d8fba4f40e8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:51.377229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb4926-5a6f-462d-b3ce-ecc46b7c7ece None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:51.377839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb4926-5a6f-462d-b3ce-ecc46b7c7ece None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:51.378315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb4926-5a6f-462d-b3ce-ecc46b7c7ece None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:51.379897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 351/703] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:51.386150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ddfa8-59ae-4ce8-aa07-edb82eb30868 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:51.386429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ddfa8-59ae-4ce8-aa07-edb82eb30868 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:51.386710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ddfa8-59ae-4ce8-aa07-edb82eb30868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:51.413228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-054ddfa8-59ae-4ce8-aa07-edb82eb30868 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:51.414543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-054ddfa8-59ae-4ce8-aa07-edb82eb30868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:51.447684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a37fb7-e42c-4406-9668-2d8fba4f40e8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:51.483712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151038c-c3f5-4e0c-9f16-f70c9c508388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:51.484444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151038c-c3f5-4e0c-9f16-f70c9c508388 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:51.484756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151038c-c3f5-4e0c-9f16-f70c9c508388 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:51.485052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151038c-c3f5-4e0c-9f16-f70c9c508388 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:51.485772 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1151038c-c3f5-4e0c-9f16-f70c9c508388 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:51.486144 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1151038c-c3f5-4e0c-9f16-f70c9c508388 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:51.487833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1151038c-c3f5-4e0c-9f16-f70c9c508388 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:51.492864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 352/704] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:51.516317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 353/705] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Sep 6 21:00:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:51.521830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ddfa8-59ae-4ce8-aa07-edb82eb30868 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:51.674923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ddfa8-59ae-4ce8-aa07-edb82eb30868 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:51.682107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:51.726861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9806f5-c7cd-4eab-b5c7-77256468ca0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:51.727506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9806f5-c7cd-4eab-b5c7-77256468ca0d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:51.727785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9806f5-c7cd-4eab-b5c7-77256468ca0d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:51.728056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9806f5-c7cd-4eab-b5c7-77256468ca0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:51.815743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9806f5-c7cd-4eab-b5c7-77256468ca0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:51.817270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9806f5-c7cd-4eab-b5c7-77256468ca0d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:51.830353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151038c-c3f5-4e0c-9f16-f70c9c508388 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:51.867264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:51.953404 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 353/706] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:51.955434 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c050d4e-b1dc-48ad-8e78-6b70a4d1febd None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:51.956275 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c050d4e-b1dc-48ad-8e78-6b70a4d1febd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:51.956719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c050d4e-b1dc-48ad-8e78-6b70a4d1febd None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:51.958560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 354/707] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:51.963095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151038c-c3f5-4e0c-9f16-f70c9c508388 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:51.963234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c050d4e-b1dc-48ad-8e78-6b70a4d1febd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:51.965683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:52.001214 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4c5be-c235-4bea-a587-aa5e30ef37dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:52.001888 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4c5be-c235-4bea-a587-aa5e30ef37dc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:52.002169 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4c5be-c235-4bea-a587-aa5e30ef37dc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:52.002461 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4c5be-c235-4bea-a587-aa5e30ef37dc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:52.003033 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4c5be-c235-4bea-a587-aa5e30ef37dc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:52.004421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4c5be-c235-4bea-a587-aa5e30ef37dc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-bbb4c5be-c235-4bea-a587-aa5e30ef37dc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:52.015645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:52.016749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 21:00:52.017591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:52.021860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 354/708] 10.222.0.79 () {56 vars in 1129 bytes} [Tue Sep 6 21:00:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:00:52.023051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fcfae-d33a-4787-a198-8b0f62296a8e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:52.023342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fcfae-d33a-4787-a198-8b0f62296a8e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:52.023618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fcfae-d33a-4787-a198-8b0f62296a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:52.043088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-211fcfae-d33a-4787-a198-8b0f62296a8e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:52.044381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-211fcfae-d33a-4787-a198-8b0f62296a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:52.088042 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c050d4e-b1dc-48ad-8e78-6b70a4d1febd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:52.134480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 355/709] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:52.135435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fcfae-d33a-4787-a198-8b0f62296a8e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:52.137487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d18341-67a3-4cbc-a8d1-74db439e83b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:52.138468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d18341-67a3-4cbc-a8d1-74db439e83b0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:52.139741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d18341-67a3-4cbc-a8d1-74db439e83b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:52.140023 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d18341-67a3-4cbc-a8d1-74db439e83b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:52.153253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d18341-67a3-4cbc-a8d1-74db439e83b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:52.155526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d18341-67a3-4cbc-a8d1-74db439e83b0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:52.156723 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c5d18341-67a3-4cbc-a8d1-74db439e83b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:52.180030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:52.181369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:52.182268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:52.182268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:52.182268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:52.185047 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 355/710] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:52.192538 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d18341-67a3-4cbc-a8d1-74db439e83b0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:52.265846 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fcfae-d33a-4787-a198-8b0f62296a8e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:52.315610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0561fd-977b-4063-bd99-ced6f62941d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:52.316432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0561fd-977b-4063-bd99-ced6f62941d4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:52.320880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0561fd-977b-4063-bd99-ced6f62941d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:52.321290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0561fd-977b-4063-bd99-ced6f62941d4 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:52.324574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0561fd-977b-4063-bd99-ced6f62941d4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:52.325196 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0561fd-977b-4063-bd99-ced6f62941d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:52.326903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0561fd-977b-4063-bd99-ced6f62941d4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:52.337209 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 356/711] 10.222.0.79 () {56 vars in 1124 bytes} [Tue Sep 6 21:00:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:52.338419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:52.346506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0561fd-977b-4063-bd99-ced6f62941d4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:52.426116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d18341-67a3-4cbc-a8d1-74db439e83b0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:52.489807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:52.497062 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff82a78-3de6-4d9b-8716-c6c3e2fb2267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:52.498190 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff82a78-3de6-4d9b-8716-c6c3e2fb2267 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:52.498529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff82a78-3de6-4d9b-8716-c6c3e2fb2267 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:52.498808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff82a78-3de6-4d9b-8716-c6c3e2fb2267 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:52.499481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff82a78-3de6-4d9b-8716-c6c3e2fb2267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:52.499785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff82a78-3de6-4d9b-8716-c6c3e2fb2267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:52.501835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff82a78-3de6-4d9b-8716-c6c3e2fb2267 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:52.533745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0561fd-977b-4063-bd99-ced6f62941d4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:52.536276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 356/712] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:52.537811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff82a78-3de6-4d9b-8716-c6c3e2fb2267 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:52.598094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6937c-5505-49ec-a45b-3363ea12df0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:52.598770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6937c-5505-49ec-a45b-3363ea12df0c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:52.599049 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6937c-5505-49ec-a45b-3363ea12df0c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:52.599385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6937c-5505-49ec-a45b-3363ea12df0c None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:52.600081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6937c-5505-49ec-a45b-3363ea12df0c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:52.600385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6937c-5505-49ec-a45b-3363ea12df0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:52.601814 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6937c-5505-49ec-a45b-3363ea12df0c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:52.613496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 357/713] 10.222.0.79 () {56 vars in 1124 bytes} [Tue Sep 6 21:00:52 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:52.614517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6937c-5505-49ec-a45b-3363ea12df0c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:52.670878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff82a78-3de6-4d9b-8716-c6c3e2fb2267 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:52.689670 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:52.710556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461efdb-0c40-4954-ac13-aed20869e36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:52.711230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461efdb-0c40-4954-ac13-aed20869e36a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:52.711507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461efdb-0c40-4954-ac13-aed20869e36a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:52.711837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461efdb-0c40-4954-ac13-aed20869e36a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:52.719727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461efdb-0c40-4954-ac13-aed20869e36a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:52.721248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461efdb-0c40-4954-ac13-aed20869e36a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0461efdb-0c40-4954-ac13-aed20869e36a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:52.735528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:52.736501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:52.737425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:52.737425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:52.737425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:52.737425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:52.737425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:52.737425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:52.737425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:52.737425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:52.737425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 357/714] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:52.743114 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461efdb-0c40-4954-ac13-aed20869e36a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:52.828090 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6937c-5505-49ec-a45b-3363ea12df0c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:52.858791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461efdb-0c40-4954-ac13-aed20869e36a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:52.888720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446e277-5ce9-4027-a796-0504838fdba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:52.890213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446e277-5ce9-4027-a796-0504838fdba5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:52.890652 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446e277-5ce9-4027-a796-0504838fdba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:52.890926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446e277-5ce9-4027-a796-0504838fdba5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:52.891900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446e277-5ce9-4027-a796-0504838fdba5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:52.893856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446e277-5ce9-4027-a796-0504838fdba5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:52.896217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c20228-ab2c-43ec-94c9-2a9d3e671427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:52.896919 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c20228-ab2c-43ec-94c9-2a9d3e671427 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:52.897257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c20228-ab2c-43ec-94c9-2a9d3e671427 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:52.897591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c20228-ab2c-43ec-94c9-2a9d3e671427 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:52.898351 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c20228-ab2c-43ec-94c9-2a9d3e671427 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:52.898717 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c20228-ab2c-43ec-94c9-2a9d3e671427 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:52.900276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c20228-ab2c-43ec-94c9-2a9d3e671427 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a446e277-5ce9-4027-a796-0504838fdba5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:52.905397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:52.906580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:52.907637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 358/715] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:52 2022] GET /identity/v3/services/network => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:52.913884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446e277-5ce9-4027-a796-0504838fdba5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:52.916426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 358/716] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:52.922486 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c20228-ab2c-43ec-94c9-2a9d3e671427 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:52.929006 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:52.965142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:53.044068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c20228-ab2c-43ec-94c9-2a9d3e671427 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:53.044283 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446e277-5ce9-4027-a796-0504838fdba5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:53.089379 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9346ce-ef97-428b-8c41-16a1b3a4ae7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:53.089908 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9346ce-ef97-428b-8c41-16a1b3a4ae7d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:53.090185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9346ce-ef97-428b-8c41-16a1b3a4ae7d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:53.090484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9346ce-ef97-428b-8c41-16a1b3a4ae7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:53.091276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9346ce-ef97-428b-8c41-16a1b3a4ae7d None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=12d51454d20c46d7a5e6d963a2f987cc) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:53.092154 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9346ce-ef97-428b-8c41-16a1b3a4ae7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:53.093731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9346ce-ef97-428b-8c41-16a1b3a4ae7d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:53.095132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b82125-1560-4b87-806c-a7f96d4fbb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:53.096386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b82125-1560-4b87-806c-a7f96d4fbb8d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:53.097595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b82125-1560-4b87-806c-a7f96d4fbb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:53.098098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b82125-1560-4b87-806c-a7f96d4fbb8d None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:53.099219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b82125-1560-4b87-806c-a7f96d4fbb8d None admin] RBAC: Adding query filter params (name=network) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:53.099725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b82125-1560-4b87-806c-a7f96d4fbb8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:53.100562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 359/717] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:52 2022] GET /identity/v3/role_assignments?user.id=12d51454d20c46d7a5e6d963a2f987cc&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=3b30504991194002b41f8587d0851c10 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:53.101608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b82125-1560-4b87-806c-a7f96d4fbb8d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:53.107703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 359/718] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Sep 6 21:00:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:53.114070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9346ce-ef97-428b-8c41-16a1b3a4ae7d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:53.232914 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:53.236794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9346ce-ef97-428b-8c41-16a1b3a4ae7d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:53.276037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20eaf52-16df-4312-b284-2b58593fb48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:53.277173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20eaf52-16df-4312-b284-2b58593fb48b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:53.277986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20eaf52-16df-4312-b284-2b58593fb48b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:53.278461 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20eaf52-16df-4312-b284-2b58593fb48b None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:53.279372 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20eaf52-16df-4312-b284-2b58593fb48b None admin] RBAC: Adding query filter params (type=network) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:53.279876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20eaf52-16df-4312-b284-2b58593fb48b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:53.281552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20eaf52-16df-4312-b284-2b58593fb48b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:53.288687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 360/719] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Sep 6 21:00:53 2022] GET /identity/v3/services?type=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:53.611323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ca99b-2c34-4c57-a501-45d9d2355fc0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:53.611962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ca99b-2c34-4c57-a501-45d9d2355fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:53.612458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ca99b-2c34-4c57-a501-45d9d2355fc0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:53.614080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 360/720] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:53.620281 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f2abf-63f5-4d15-9039-d65764e4c44b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:53.620578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f2abf-63f5-4d15-9039-d65764e4c44b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:53.620860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f2abf-63f5-4d15-9039-d65764e4c44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:53.650437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-984f2abf-63f5-4d15-9039-d65764e4c44b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:53.651640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-984f2abf-63f5-4d15-9039-d65764e4c44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:53.748542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 361/721] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:53.855707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793214-6668-4760-ae2f-f4d0147a0eb1 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:53.856387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793214-6668-4760-ae2f-f4d0147a0eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:53.856940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793214-6668-4760-ae2f-f4d0147a0eb1 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:53.858750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 361/722] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:53.864363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d6c70-5e1b-4b75-ba9b-9c98f1ad5183 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:53.864649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d6c70-5e1b-4b75-ba9b-9c98f1ad5183 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:53.864943 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d6c70-5e1b-4b75-ba9b-9c98f1ad5183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:53.888220 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0f6d6c70-5e1b-4b75-ba9b-9c98f1ad5183 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:53.889443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0f6d6c70-5e1b-4b75-ba9b-9c98f1ad5183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:53.976562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 362/723] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:54.054013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8793214-6668-4760-ae2f-f4d0147a0eb1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:54.137912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:54.170202 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8793214-6668-4760-ae2f-f4d0147a0eb1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:54.205762 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb35a8f-ac40-4015-a15b-b965440c6c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:54.206395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb35a8f-ac40-4015-a15b-b965440c6c07 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:54.206670 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb35a8f-ac40-4015-a15b-b965440c6c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:54.206941 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb35a8f-ac40-4015-a15b-b965440c6c07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:54.217533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb35a8f-ac40-4015-a15b-b965440c6c07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ffb35a8f-ac40-4015-a15b-b965440c6c07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:54.222430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:54.223331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:54.224157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:54.224157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:54.224157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:54.224157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:54.224157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:54.224157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:54.224157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:54.224157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:54.224157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:54.224157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:54.224157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:00:54.224157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:54.224157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 362/724] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:00:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:54.229983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d6c70-5e1b-4b75-ba9b-9c98f1ad5183 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:54.275160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:54.295703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb35a8f-ac40-4015-a15b-b965440c6c07 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:54.414691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb35a8f-ac40-4015-a15b-b965440c6c07 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:54.446962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d6c70-5e1b-4b75-ba9b-9c98f1ad5183 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:54.448932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:54.461046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcd9e3-bb03-44b5-849d-4e82b1bc8bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:54.462393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcd9e3-bb03-44b5-849d-4e82b1bc8bdd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:54.462759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcd9e3-bb03-44b5-849d-4e82b1bc8bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:54.463070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcd9e3-bb03-44b5-849d-4e82b1bc8bdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:54.483535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcd9e3-bb03-44b5-849d-4e82b1bc8bdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-27fcd9e3-bb03-44b5-849d-4e82b1bc8bdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:54.489448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:54.491627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:54.493626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:54.493626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:54.493626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:54.493626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:54.493626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:54.493626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:54.493626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:54.493626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:54.493626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:54.493626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:54.493626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:00:54.493626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:54.493626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 363/725] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:00:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:00:54.500677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcd9e3-bb03-44b5-849d-4e82b1bc8bdd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:54.526650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4ab86-d6dc-4cc8-b9c1-5546f87ee9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:54.527381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4ab86-d6dc-4cc8-b9c1-5546f87ee9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:54.527663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4ab86-d6dc-4cc8-b9c1-5546f87ee9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:54.527969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4ab86-d6dc-4cc8-b9c1-5546f87ee9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:54.528672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb4ab86-d6dc-4cc8-b9c1-5546f87ee9e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:54.528976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb4ab86-d6dc-4cc8-b9c1-5546f87ee9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:54.530595 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb4ab86-d6dc-4cc8-b9c1-5546f87ee9e5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:54.549320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 363/726] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:54.555318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4ab86-d6dc-4cc8-b9c1-5546f87ee9e5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:54.620155 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcd9e3-bb03-44b5-849d-4e82b1bc8bdd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:54.659843 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09315b91-8023-4f18-b23e-40f1130a9e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:54.660464 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09315b91-8023-4f18-b23e-40f1130a9e7b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:54.660767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09315b91-8023-4f18-b23e-40f1130a9e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:54.661040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09315b91-8023-4f18-b23e-40f1130a9e7b None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:54.662215 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09315b91-8023-4f18-b23e-40f1130a9e7b None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:54.662521 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09315b91-8023-4f18-b23e-40f1130a9e7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:54.664195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09315b91-8023-4f18-b23e-40f1130a9e7b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:54.674036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 364/727] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:00:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:00:54.680370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09315b91-8023-4f18-b23e-40f1130a9e7b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:54.742872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:54.772033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4ab86-d6dc-4cc8-b9c1-5546f87ee9e5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:54.796770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:54.843508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e8f64-2c52-427a-b5e2-de8e2b44a5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:54.844557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e8f64-2c52-427a-b5e2-de8e2b44a5ba None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:54.845046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e8f64-2c52-427a-b5e2-de8e2b44a5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:54.845491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e8f64-2c52-427a-b5e2-de8e2b44a5ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:54.859508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e8f64-2c52-427a-b5e2-de8e2b44a5ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:54.861863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e8f64-2c52-427a-b5e2-de8e2b44a5ba None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-800e8f64-2c52-427a-b5e2-de8e2b44a5ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:54.880829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:54.881799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:54.882687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:54.882687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:54.882687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:54.886639 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 364/728] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:54.887950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e8f64-2c52-427a-b5e2-de8e2b44a5ba None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:54.901969 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09315b91-8023-4f18-b23e-40f1130a9e7b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:54.969108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fe70c-bd21-489e-a910-4dfcab0dcb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:54.969675 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fe70c-bd21-489e-a910-4dfcab0dcb77 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:54.969955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fe70c-bd21-489e-a910-4dfcab0dcb77 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:54.970225 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fe70c-bd21-489e-a910-4dfcab0dcb77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:54.994330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fe70c-bd21-489e-a910-4dfcab0dcb77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:54.995812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fe70c-bd21-489e-a910-4dfcab0dcb77 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0c4fe70c-bd21-489e-a910-4dfcab0dcb77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:55.027152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:55.028217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:55.029151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:55.029151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:55.029151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:55.034353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 365/729] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:55.037369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fe70c-bd21-489e-a910-4dfcab0dcb77 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:55.081989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e8f64-2c52-427a-b5e2-de8e2b44a5ba None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:55.150337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32553fe4-1a29-4d82-910f-8f895ed8d770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:55.150960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32553fe4-1a29-4d82-910f-8f895ed8d770 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:55.151238 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32553fe4-1a29-4d82-910f-8f895ed8d770 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:55.151510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32553fe4-1a29-4d82-910f-8f895ed8d770 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:55.152168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32553fe4-1a29-4d82-910f-8f895ed8d770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:55.152470 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32553fe4-1a29-4d82-910f-8f895ed8d770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:55.154105 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32553fe4-1a29-4d82-910f-8f895ed8d770 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:55.171764 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:55.176046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 365/730] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:55.183982 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32553fe4-1a29-4d82-910f-8f895ed8d770 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:55.243502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fe70c-bd21-489e-a910-4dfcab0dcb77 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:55.255792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:55.319773 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21366afb-bc88-4aac-8993-a45cd95c86ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:55.320394 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21366afb-bc88-4aac-8993-a45cd95c86ce None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:55.320690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21366afb-bc88-4aac-8993-a45cd95c86ce None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:55.320968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21366afb-bc88-4aac-8993-a45cd95c86ce None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:55.321622 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21366afb-bc88-4aac-8993-a45cd95c86ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:55.321923 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21366afb-bc88-4aac-8993-a45cd95c86ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:55.328942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21366afb-bc88-4aac-8993-a45cd95c86ce None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:55.354980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 366/731] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:55.361747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21366afb-bc88-4aac-8993-a45cd95c86ce None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:55.365974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32553fe4-1a29-4d82-910f-8f895ed8d770 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:55.405366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cbd1f-39f8-4971-a551-1071f4057781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:55.406006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cbd1f-39f8-4971-a551-1071f4057781 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:55.406284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cbd1f-39f8-4971-a551-1071f4057781 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:55.406554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cbd1f-39f8-4971-a551-1071f4057781 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:55.420698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cbd1f-39f8-4971-a551-1071f4057781 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:55.422859 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cbd1f-39f8-4971-a551-1071f4057781 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ad4cbd1f-39f8-4971-a551-1071f4057781 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:55.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:55.471786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:55.472763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:55.472763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:55.472763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:55.472763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:55.472763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:55.472763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:55.472763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:55.472763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:55.472763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 21:00:55.472763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:55.474769 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 366/732] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:00:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:55.479422 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cbd1f-39f8-4971-a551-1071f4057781 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:55.537588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21366afb-bc88-4aac-8993-a45cd95c86ce None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:55.591562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:55.606651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5338e-8821-4e57-b0c9-4d1475016912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:55.607299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5338e-8821-4e57-b0c9-4d1475016912 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:55.607585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5338e-8821-4e57-b0c9-4d1475016912 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:55.607860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5338e-8821-4e57-b0c9-4d1475016912 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:55.617758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5338e-8821-4e57-b0c9-4d1475016912 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:55.619553 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5338e-8821-4e57-b0c9-4d1475016912 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0cc5338e-8821-4e57-b0c9-4d1475016912 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:55.635556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:55.636637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:55.638347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:55.638347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:55.638347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:55.638347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:55.638347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:55.638347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:55.638347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:55.638347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:55.638347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 21:00:55.638347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:55.640741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 367/733] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Sep 6 21:00:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:55.642377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bf081-679f-4a5f-8762-b9353b7bd897 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:55.642666 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bf081-679f-4a5f-8762-b9353b7bd897 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:55.642936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bf081-679f-4a5f-8762-b9353b7bd897 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:55.644363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 368/734] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:55.646646 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bf081-679f-4a5f-8762-b9353b7bd897 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:55.652411 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cbd1f-39f8-4971-a551-1071f4057781 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:55.689956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fa43a-01cf-4450-ab49-84789a0e37e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:55.690588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fa43a-01cf-4450-ab49-84789a0e37e8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:55.690955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fa43a-01cf-4450-ab49-84789a0e37e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:55.691325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fa43a-01cf-4450-ab49-84789a0e37e8 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:55.692065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fa43a-01cf-4450-ab49-84789a0e37e8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:55.692370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fa43a-01cf-4450-ab49-84789a0e37e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:55.693147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:55.693871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fa43a-01cf-4450-ab49-84789a0e37e8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:55.718695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 367/735] 10.222.0.79 () {56 vars in 1135 bytes} [Tue Sep 6 21:00:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:55.718695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74173fc2-e314-4655-a3a5-49de84a5c5ec None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:55.718695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74173fc2-e314-4655-a3a5-49de84a5c5ec None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:55.718695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74173fc2-e314-4655-a3a5-49de84a5c5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:55.737434 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-74173fc2-e314-4655-a3a5-49de84a5c5ec None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:55.738640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-74173fc2-e314-4655-a3a5-49de84a5c5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:55.775999 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bf081-679f-4a5f-8762-b9353b7bd897 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:55.816061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc301fe5-fd67-47e0-b454-1b26946999d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:55.816598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc301fe5-fd67-47e0-b454-1b26946999d8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:55.816893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc301fe5-fd67-47e0-b454-1b26946999d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:55.817159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc301fe5-fd67-47e0-b454-1b26946999d8 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:55.817972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc301fe5-fd67-47e0-b454-1b26946999d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:55.818284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc301fe5-fd67-47e0-b454-1b26946999d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:55.819731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc301fe5-fd67-47e0-b454-1b26946999d8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:55.824999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 368/736] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:55.827074 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3e432-937d-44f4-a9a2-e5431cf951bb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:55.827906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3e432-937d-44f4-a9a2-e5431cf951bb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:55.828400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3e432-937d-44f4-a9a2-e5431cf951bb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:55.835971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 369/737] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:55.838260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3e432-937d-44f4-a9a2-e5431cf951bb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:55.840966 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 369/738] 10.222.0.79 () {56 vars in 1149 bytes} [Tue Sep 6 21:00:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:55.842923 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d8a35-1958-4fdf-9e22-493350a5ff6a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:55.843272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d8a35-1958-4fdf-9e22-493350a5ff6a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:55.843490 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d8a35-1958-4fdf-9e22-493350a5ff6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:55.866368 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e78d8a35-1958-4fdf-9e22-493350a5ff6a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:55.867844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e78d8a35-1958-4fdf-9e22-493350a5ff6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:55.902301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:55.945060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 370/739] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:55.946210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d8a35-1958-4fdf-9e22-493350a5ff6a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:55.962352 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3e432-937d-44f4-a9a2-e5431cf951bb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:55.969411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:56.001458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ff56f-ee7f-4e61-8b7a-da366ea590e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:56.002390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ff56f-ee7f-4e61-8b7a-da366ea590e5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:56.002856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ff56f-ee7f-4e61-8b7a-da366ea590e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:56.003305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ff56f-ee7f-4e61-8b7a-da366ea590e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:56.055214 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d8a35-1958-4fdf-9e22-493350a5ff6a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:56.076039 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ff56f-ee7f-4e61-8b7a-da366ea590e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:56.077890 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ff56f-ee7f-4e61-8b7a-da366ea590e5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:56.090422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8450a7-e981-436a-8e72-0935ad6bde13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:56.091036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8450a7-e981-436a-8e72-0935ad6bde13 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:56.091367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8450a7-e981-436a-8e72-0935ad6bde13 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:56.091637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8450a7-e981-436a-8e72-0935ad6bde13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:56.128625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:56.164497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8450a7-e981-436a-8e72-0935ad6bde13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:56.166064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8450a7-e981-436a-8e72-0935ad6bde13 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:56.181240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:56.193706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 370/740] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:56.193706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e9139-6960-4672-b2a1-20a8bf382ef4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:56.193706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e9139-6960-4672-b2a1-20a8bf382ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:56.193706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e9139-6960-4672-b2a1-20a8bf382ef4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:56.193706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 371/741] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:56.193706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e9139-6960-4672-b2a1-20a8bf382ef4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:56.279864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 371/742] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:00:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 21:00:56.280755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8450a7-e981-436a-8e72-0935ad6bde13 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:56.296806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e9139-6960-4672-b2a1-20a8bf382ef4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:56.333059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf127a7-4ac2-4fe7-a22a-d61fc210a148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:56.333587 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf127a7-4ac2-4fe7-a22a-d61fc210a148 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:56.333864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf127a7-4ac2-4fe7-a22a-d61fc210a148 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:56.334137 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf127a7-4ac2-4fe7-a22a-d61fc210a148 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:56.334793 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf127a7-4ac2-4fe7-a22a-d61fc210a148 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:56.336197 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf127a7-4ac2-4fe7-a22a-d61fc210a148 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:56.352642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 372/743] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 21:00:56 2022] POST /identity/v3/services => generated 251 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:00:56.353890 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eddead-3f15-4b0d-a978-1a91dac81c53 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:56.354258 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eddead-3f15-4b0d-a978-1a91dac81c53 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:56.354608 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eddead-3f15-4b0d-a978-1a91dac81c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:56.369973 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:56.377614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-11eddead-3f15-4b0d-a978-1a91dac81c53 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:56.380931 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-11eddead-3f15-4b0d-a978-1a91dac81c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:56.394392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8450a7-e981-436a-8e72-0935ad6bde13 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:56.410171 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:56.448766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6e233-065f-48a1-9f4a-731408b199c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:56.449766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6e233-065f-48a1-9f4a-731408b199c5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:56.450063 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6e233-065f-48a1-9f4a-731408b199c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:56.450339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6e233-065f-48a1-9f4a-731408b199c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:56.463079 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 373/744] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 21:00:56.463978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eddead-3f15-4b0d-a978-1a91dac81c53 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:56.477368 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6e233-065f-48a1-9f4a-731408b199c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:56.479432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6e233-065f-48a1-9f4a-731408b199c5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-70e6e233-065f-48a1-9f4a-731408b199c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:56.497763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:56.498817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:56.499763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:56.499763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:56.499763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:56.506992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 372/745] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:56.508299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6e233-065f-48a1-9f4a-731408b199c5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:56.580704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eddead-3f15-4b0d-a978-1a91dac81c53 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:56.633425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7007c-5f52-451d-aa38-eb58d7eee473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:56.634160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7007c-5f52-451d-aa38-eb58d7eee473 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:56.634436 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7007c-5f52-451d-aa38-eb58d7eee473 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:56.634706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7007c-5f52-451d-aa38-eb58d7eee473 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:56.638153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:56.648584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7007c-5f52-451d-aa38-eb58d7eee473 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-fce7007c-5f52-451d-aa38-eb58d7eee473 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:00:56.655130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:56.656149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:56.657021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:56.657021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:56.657021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:56.657021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:56.657021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:00:56.657021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:00:56.657021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:00:56.657021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:00:56.657021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:00:56.657021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:00:56.657021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:00:56.657021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:56.657021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 374/746] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:00:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:56.657716 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7007c-5f52-451d-aa38-eb58d7eee473 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:56.758165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6e233-065f-48a1-9f4a-731408b199c5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:56.766128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7007c-5f52-451d-aa38-eb58d7eee473 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:56.795764 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a2dd3-eaf2-43e1-aee4-725db506c917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:56.796408 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a2dd3-eaf2-43e1-aee4-725db506c917 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:56.796729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a2dd3-eaf2-43e1-aee4-725db506c917 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:56.797028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a2dd3-eaf2-43e1-aee4-725db506c917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:56.801691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a6795-cd8b-443f-b0d1-05d1b38459db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:56.802515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a6795-cd8b-443f-b0d1-05d1b38459db None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:56.802972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a6795-cd8b-443f-b0d1-05d1b38459db None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:56.803471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a6795-cd8b-443f-b0d1-05d1b38459db None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:56.804292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a6795-cd8b-443f-b0d1-05d1b38459db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:56.804809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a6795-cd8b-443f-b0d1-05d1b38459db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:56.806574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a6795-cd8b-443f-b0d1-05d1b38459db None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:56.807635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a2dd3-eaf2-43e1-aee4-725db506c917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:56.809150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a2dd3-eaf2-43e1-aee4-725db506c917 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:56.824479 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 375/747] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:56.825708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a6795-cd8b-443f-b0d1-05d1b38459db None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1b2a2dd3-eaf2-43e1-aee4-725db506c917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:56.830446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:56.831446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:56.832523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:56.832523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:56.832523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:56.839620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 373/748] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:56.840573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a2dd3-eaf2-43e1-aee4-725db506c917 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:56.858866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:56.889161 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:56.936198 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a6795-cd8b-443f-b0d1-05d1b38459db None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:56.965695 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a2dd3-eaf2-43e1-aee4-725db506c917 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:56.971875 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b25f1f-8847-4d49-8a8a-6e66c4fc1592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:56.972797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b25f1f-8847-4d49-8a8a-6e66c4fc1592 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:56.973111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b25f1f-8847-4d49-8a8a-6e66c4fc1592 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:56.973796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b25f1f-8847-4d49-8a8a-6e66c4fc1592 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:56.974483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b25f1f-8847-4d49-8a8a-6e66c4fc1592 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:56.974789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b25f1f-8847-4d49-8a8a-6e66c4fc1592 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:56.976836 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b25f1f-8847-4d49-8a8a-6e66c4fc1592 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:56.991961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 376/749] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:00:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:56.991961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b25f1f-8847-4d49-8a8a-6e66c4fc1592 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:57.008738 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce07d1-8256-4621-93bf-d8acd7cb3e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:57.009444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce07d1-8256-4621-93bf-d8acd7cb3e4f None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:57.009688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce07d1-8256-4621-93bf-d8acd7cb3e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:57.009891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce07d1-8256-4621-93bf-d8acd7cb3e4f None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:57.010544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce07d1-8256-4621-93bf-d8acd7cb3e4f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:57.012195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce07d1-8256-4621-93bf-d8acd7cb3e4f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:57.032301 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 374/750] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 21:00:56 2022] POST /identity/v3/services => generated 240 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:00:57.033416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce07d1-8256-4621-93bf-d8acd7cb3e4f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:57.109208 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b25f1f-8847-4d49-8a8a-6e66c4fc1592 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:57.130580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:57.156943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:57.158548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53623c3b-05ca-403b-b7a6-4e05b51f2628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:57.159297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53623c3b-05ca-403b-b7a6-4e05b51f2628 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:57.159681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53623c3b-05ca-403b-b7a6-4e05b51f2628 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:57.160101 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53623c3b-05ca-403b-b7a6-4e05b51f2628 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:57.171350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53623c3b-05ca-403b-b7a6-4e05b51f2628 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:57.173016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53623c3b-05ca-403b-b7a6-4e05b51f2628 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-53623c3b-05ca-403b-b7a6-4e05b51f2628 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:57.187852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:57.188865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:57.189782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:57.189782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:57.189782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:57.189782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:57.189782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:57.189782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:57.189782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:57.189782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:57.194523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 377/751] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:57.195533 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53623c3b-05ca-403b-b7a6-4e05b51f2628 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:57.197972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce07d1-8256-4621-93bf-d8acd7cb3e4f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:57.247003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17f2aa-06dc-4210-a2ac-bfcd4d863ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:57.248088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17f2aa-06dc-4210-a2ac-bfcd4d863ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:57.248438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17f2aa-06dc-4210-a2ac-bfcd4d863ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:57.249541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17f2aa-06dc-4210-a2ac-bfcd4d863ee3 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:57.249541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17f2aa-06dc-4210-a2ac-bfcd4d863ee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:57.249724 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17f2aa-06dc-4210-a2ac-bfcd4d863ee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:57.251059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17f2aa-06dc-4210-a2ac-bfcd4d863ee3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:57.264799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 375/752] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:57.265649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17f2aa-06dc-4210-a2ac-bfcd4d863ee3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:57.324982 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53623c3b-05ca-403b-b7a6-4e05b51f2628 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:57.360771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417753f5-4d9e-4f26-9ad6-bd8d778b5e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:57.361727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417753f5-4d9e-4f26-9ad6-bd8d778b5e5b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:57.362184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417753f5-4d9e-4f26-9ad6-bd8d778b5e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:57.362633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417753f5-4d9e-4f26-9ad6-bd8d778b5e5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:57.374126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417753f5-4d9e-4f26-9ad6-bd8d778b5e5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:57.375883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417753f5-4d9e-4f26-9ad6-bd8d778b5e5b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:57.387687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17f2aa-06dc-4210-a2ac-bfcd4d863ee3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-417753f5-4d9e-4f26-9ad6-bd8d778b5e5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:57.391233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:57.392146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:57.393083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:57.393083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:57.393083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:57.395281 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 378/753] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:57.396259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417753f5-4d9e-4f26-9ad6-bd8d778b5e5b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:57.425103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b309be-8924-41dc-83c3-449e22dad6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:57.425841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b309be-8924-41dc-83c3-449e22dad6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:57.426157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b309be-8924-41dc-83c3-449e22dad6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:57.426492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b309be-8924-41dc-83c3-449e22dad6c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:57.427286 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b309be-8924-41dc-83c3-449e22dad6c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:57.427630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b309be-8924-41dc-83c3-449e22dad6c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:57.429193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b309be-8924-41dc-83c3-449e22dad6c5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:57.430939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:57.444104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 376/754] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:57.445274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b309be-8924-41dc-83c3-449e22dad6c5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:57.449430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:57.517465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417753f5-4d9e-4f26-9ad6-bd8d778b5e5b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:57.552800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7052a25-eb50-43e9-bda3-8dd9c212e3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:57.553684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7052a25-eb50-43e9-bda3-8dd9c212e3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:57.554188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7052a25-eb50-43e9-bda3-8dd9c212e3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:57.554677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7052a25-eb50-43e9-bda3-8dd9c212e3d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:57.563708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7052a25-eb50-43e9-bda3-8dd9c212e3d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:57.565554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7052a25-eb50-43e9-bda3-8dd9c212e3d2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f7052a25-eb50-43e9-bda3-8dd9c212e3d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:57.577550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:57.578473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:57.579357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:57.579357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:57.579357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:57.579357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:57.579357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:57.579357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:57.579357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:57.579357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:57.580310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b309be-8924-41dc-83c3-449e22dad6c5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:57.586465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 379/755] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:57.587806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7052a25-eb50-43e9-bda3-8dd9c212e3d2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:57.629339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f5d34-6a6e-4b57-a69e-2b3fc4ae7832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:57.630009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f5d34-6a6e-4b57-a69e-2b3fc4ae7832 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:57.630405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f5d34-6a6e-4b57-a69e-2b3fc4ae7832 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:57.630775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f5d34-6a6e-4b57-a69e-2b3fc4ae7832 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:57.631526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f5d34-6a6e-4b57-a69e-2b3fc4ae7832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:57.631928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f5d34-6a6e-4b57-a69e-2b3fc4ae7832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:57.633446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f5d34-6a6e-4b57-a69e-2b3fc4ae7832 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:57.649595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 377/756] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:57.650479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f5d34-6a6e-4b57-a69e-2b3fc4ae7832 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:57.701925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:57.712987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:57.719409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7052a25-eb50-43e9-bda3-8dd9c212e3d2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:57.763864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd392c-b67f-41e6-9e87-fe5538eef992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:57.765065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd392c-b67f-41e6-9e87-fe5538eef992 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:57.765789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd392c-b67f-41e6-9e87-fe5538eef992 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:57.766247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd392c-b67f-41e6-9e87-fe5538eef992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:57.767787 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd392c-b67f-41e6-9e87-fe5538eef992 None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=061c584de8264868af8782402a98a72e) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:57.768273 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd392c-b67f-41e6-9e87-fe5538eef992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:57.769526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f5d34-6a6e-4b57-a69e-2b3fc4ae7832 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:57.770195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd392c-b67f-41e6-9e87-fe5538eef992 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:57.778662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 380/757] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:57 2022] GET /identity/v3/role_assignments?user.id=061c584de8264868af8782402a98a72e&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=3b30504991194002b41f8587d0851c10 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:57.780199 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd392c-b67f-41e6-9e87-fe5538eef992 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:57.808920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7da8e-bee4-400b-bbd7-ce872e9c8f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:57.809704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7da8e-bee4-400b-bbd7-ce872e9c8f55 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:57.810045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7da8e-bee4-400b-bbd7-ce872e9c8f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:57.810731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7da8e-bee4-400b-bbd7-ce872e9c8f55 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:57.811592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7da8e-bee4-400b-bbd7-ce872e9c8f55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:57.811934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7da8e-bee4-400b-bbd7-ce872e9c8f55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:57.813496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7da8e-bee4-400b-bbd7-ce872e9c8f55 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:57.832299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 378/758] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:57.838126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7da8e-bee4-400b-bbd7-ce872e9c8f55 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:57.921188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd392c-b67f-41e6-9e87-fe5538eef992 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:57.967457 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3e765-ed47-45cc-96a2-8932425bf10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:57.969534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3e765-ed47-45cc-96a2-8932425bf10b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:57.971344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3e765-ed47-45cc-96a2-8932425bf10b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:57.971860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3e765-ed47-45cc-96a2-8932425bf10b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:57.991407 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3e765-ed47-45cc-96a2-8932425bf10b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:57.994588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3e765-ed47-45cc-96a2-8932425bf10b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-35c3e765-ed47-45cc-96a2-8932425bf10b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:00:58.018507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:58.019542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:58.021144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:58.021144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:58.021144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:00:58.021144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:00:58.021144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:00:58.021144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:00:58.021144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:00:58.021144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:00:58.021144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:00:58.021144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:58.021144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 381/759] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Sep 6 21:00:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:58.025779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3e765-ed47-45cc-96a2-8932425bf10b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:58.051668 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:58.060997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7da8e-bee4-400b-bbd7-ce872e9c8f55 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:58.077539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:58.114284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fafc5-709f-46ea-be99-518f23c51d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:58.115260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fafc5-709f-46ea-be99-518f23c51d04 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:58.115732 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fafc5-709f-46ea-be99-518f23c51d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:58.116191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fafc5-709f-46ea-be99-518f23c51d04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:58.117176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fafc5-709f-46ea-be99-518f23c51d04 None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=c09b4900d97b4b35b617af9a42baf9ee) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:58.117700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fafc5-709f-46ea-be99-518f23c51d04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:58.119366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fafc5-709f-46ea-be99-518f23c51d04 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:58.139175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 379/760] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:00:57 2022] GET /identity/v3/role_assignments?user.id=c09b4900d97b4b35b617af9a42baf9ee&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:58.158626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3e765-ed47-45cc-96a2-8932425bf10b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:58.195948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c36cc-8230-460e-a754-2894484f3e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:58.196493 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c36cc-8230-460e-a754-2894484f3e8a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:58.196808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c36cc-8230-460e-a754-2894484f3e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:58.197272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c36cc-8230-460e-a754-2894484f3e8a None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:58.198084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c36cc-8230-460e-a754-2894484f3e8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:58.198450 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c36cc-8230-460e-a754-2894484f3e8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:58.200075 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c36cc-8230-460e-a754-2894484f3e8a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:58.232086 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 382/761] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Sep 6 21:00:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:58.237567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fafc5-709f-46ea-be99-518f23c51d04 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:58.422778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fafc5-709f-46ea-be99-518f23c51d04 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:58.465368 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391883ae-cc37-4ccd-814d-8d14fb9c30ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:58.466359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391883ae-cc37-4ccd-814d-8d14fb9c30ed None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:58.466851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391883ae-cc37-4ccd-814d-8d14fb9c30ed None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:58.467323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391883ae-cc37-4ccd-814d-8d14fb9c30ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:58.478688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391883ae-cc37-4ccd-814d-8d14fb9c30ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:58.480456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391883ae-cc37-4ccd-814d-8d14fb9c30ed None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-391883ae-cc37-4ccd-814d-8d14fb9c30ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:00:58.497220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:58.498218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:00:58.499104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:00:58.499104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:00:58.499104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:58.501304 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 380/762] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:58.508775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c36cc-8230-460e-a754-2894484f3e8a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:58.644570 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:58.682001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c36cc-8230-460e-a754-2894484f3e8a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:58.730004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94e008-ac15-49f5-b4d2-2a7394094826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:58.730543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94e008-ac15-49f5-b4d2-2a7394094826 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:58.730870 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94e008-ac15-49f5-b4d2-2a7394094826 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:58.731145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94e008-ac15-49f5-b4d2-2a7394094826 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:58.731797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94e008-ac15-49f5-b4d2-2a7394094826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:58.732100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94e008-ac15-49f5-b4d2-2a7394094826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:58.733530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94e008-ac15-49f5-b4d2-2a7394094826 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:58.754877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 383/763] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:58.760616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391883ae-cc37-4ccd-814d-8d14fb9c30ed None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:58.818691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:58.837294 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdb9e0-c5ec-453f-9ee9-543a1c5ba8a5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:58.837591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdb9e0-c5ec-453f-9ee9-543a1c5ba8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:58.837871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdb9e0-c5ec-453f-9ee9-543a1c5ba8a5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:58.839375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 384/764] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:00:58.860397 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ab55f-7ac4-40a0-9669-8a7c76eff692 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:58.860699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ab55f-7ac4-40a0-9669-8a7c76eff692 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:58.860986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ab55f-7ac4-40a0-9669-8a7c76eff692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:58.887724 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ad2ab55f-7ac4-40a0-9669-8a7c76eff692 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:58.889031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ad2ab55f-7ac4-40a0-9669-8a7c76eff692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:58.938350 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391883ae-cc37-4ccd-814d-8d14fb9c30ed None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:58.971010 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 385/765] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:00:58.993030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086b67e-f943-4642-bfca-377c6515746e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:58.994085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086b67e-f943-4642-bfca-377c6515746e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:58.994370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086b67e-f943-4642-bfca-377c6515746e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:58.994637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086b67e-f943-4642-bfca-377c6515746e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:59.003048 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086b67e-f943-4642-bfca-377c6515746e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:59.004579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086b67e-f943-4642-bfca-377c6515746e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-5086b67e-f943-4642-bfca-377c6515746e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:00:59.021961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:59.022942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:59.023996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:00:59.023996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:00:59.023996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:00:59.023996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:00:59.023996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:00:59.023996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:00:59.023996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:00:59.023996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:00:59.023996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 381/766] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:00:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:00:59.028952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ab55f-7ac4-40a0-9669-8a7c76eff692 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:59.129093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:59.163193 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ab55f-7ac4-40a0-9669-8a7c76eff692 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:59.222606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc9764-5120-4dae-a6af-e4b3d52fc82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:59.223232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc9764-5120-4dae-a6af-e4b3d52fc82f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:59.223515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc9764-5120-4dae-a6af-e4b3d52fc82f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:59.223785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc9764-5120-4dae-a6af-e4b3d52fc82f None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:59.224489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc9764-5120-4dae-a6af-e4b3d52fc82f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:59.224816 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc9764-5120-4dae-a6af-e4b3d52fc82f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:59.226263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc9764-5120-4dae-a6af-e4b3d52fc82f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:59.243632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 386/767] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:00:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:59.249095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086b67e-f943-4642-bfca-377c6515746e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:59.274418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc9764-5120-4dae-a6af-e4b3d52fc82f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:59.362661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:59.374217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086b67e-f943-4642-bfca-377c6515746e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:59.431766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81434274-b34f-42a8-b0b8-4061c4270232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:59.432452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81434274-b34f-42a8-b0b8-4061c4270232 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:59.432452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81434274-b34f-42a8-b0b8-4061c4270232 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:59.432789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81434274-b34f-42a8-b0b8-4061c4270232 None admin] PATH_INFO: `/v3/projects/05af249ba0a047fe83aefb078e203e53/users/12d51454d20c46d7a5e6d963a2f987cc/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:59.434723 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81434274-b34f-42a8-b0b8-4061c4270232 None admin] RBAC: Authorizing `identity:create_grant(project_id=05af249ba0a047fe83aefb078e203e53, user_id=12d51454d20c46d7a5e6d963a2f987cc, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:59.436514 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81434274-b34f-42a8-b0b8-4061c4270232 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:59.457901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc9764-5120-4dae-a6af-e4b3d52fc82f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:59.468901 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 382/768] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:00:59 2022] PUT /identity/v3/projects/05af249ba0a047fe83aefb078e203e53/users/12d51454d20c46d7a5e6d963a2f987cc/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:00:59.481954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:59.510395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a56f12-ee6d-4f39-a400-fa31ef6b49e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:59.511072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a56f12-ee6d-4f39-a400-fa31ef6b49e4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:59.511359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a56f12-ee6d-4f39-a400-fa31ef6b49e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:59.511629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a56f12-ee6d-4f39-a400-fa31ef6b49e4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:59.512293 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a56f12-ee6d-4f39-a400-fa31ef6b49e4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:59.513784 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a56f12-ee6d-4f39-a400-fa31ef6b49e4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-38a56f12-ee6d-4f39-a400-fa31ef6b49e4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:00:59.529632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:00:59.531038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:00:59.531876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 387/769] 10.222.0.79 () {56 vars in 1129 bytes} [Tue Sep 6 21:00:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:00:59.538957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81434274-b34f-42a8-b0b8-4061c4270232 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:59.703089 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81434274-b34f-42a8-b0b8-4061c4270232 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:59.741170 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f3382-f682-4a40-96bc-2c9f2292fa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:59.741789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f3382-f682-4a40-96bc-2c9f2292fa79 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:59.742069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f3382-f682-4a40-96bc-2c9f2292fa79 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:59.742341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f3382-f682-4a40-96bc-2c9f2292fa79 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:59.743193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f3382-f682-4a40-96bc-2c9f2292fa79 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:59.743606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f3382-f682-4a40-96bc-2c9f2292fa79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:59.745013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f3382-f682-4a40-96bc-2c9f2292fa79 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:59.751082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 383/770] 10.222.0.79 () {56 vars in 1124 bytes} [Tue Sep 6 21:00:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:59.756751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a56f12-ee6d-4f39-a400-fa31ef6b49e4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:00:59.773584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958bbc4-afd5-4ba1-b81b-0b0c12cb4bb3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:59.773866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958bbc4-afd5-4ba1-b81b-0b0c12cb4bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:59.774147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958bbc4-afd5-4ba1-b81b-0b0c12cb4bb3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:59.776538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 384/771] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:00:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:00:59.782316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64146ba-64bb-48bc-974c-a082cf6a3259 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:59.782600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64146ba-64bb-48bc-974c-a082cf6a3259 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:59.782930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64146ba-64bb-48bc-974c-a082cf6a3259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:59.806983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e64146ba-64bb-48bc-974c-a082cf6a3259 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:00:59.808192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e64146ba-64bb-48bc-974c-a082cf6a3259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:00:59.820246 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:00:59.911594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a56f12-ee6d-4f39-a400-fa31ef6b49e4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:00:59.931734 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 385/772] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:00:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:00:59.974326 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828558e0-8815-4f0b-8e8b-23e65cc67496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:00:59.975032 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828558e0-8815-4f0b-8e8b-23e65cc67496 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:00:59.975291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828558e0-8815-4f0b-8e8b-23e65cc67496 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:00:59.975567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828558e0-8815-4f0b-8e8b-23e65cc67496 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:00:59.976221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828558e0-8815-4f0b-8e8b-23e65cc67496 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:00:59.976527 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828558e0-8815-4f0b-8e8b-23e65cc67496 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:00:59.978069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828558e0-8815-4f0b-8e8b-23e65cc67496 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:00:59.988996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 388/773] 10.222.0.79 () {56 vars in 1124 bytes} [Tue Sep 6 21:00:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:00:59.989665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64146ba-64bb-48bc-974c-a082cf6a3259 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:00.126435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64146ba-64bb-48bc-974c-a082cf6a3259 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:00.165874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ca798-67a2-424d-8783-76dc80b7fb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:00.166500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ca798-67a2-424d-8783-76dc80b7fb9b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:00.166780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ca798-67a2-424d-8783-76dc80b7fb9b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:00.167054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ca798-67a2-424d-8783-76dc80b7fb9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:00.167767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ca798-67a2-424d-8783-76dc80b7fb9b None admin] RBAC: Adding query filter params (interface=public, service_id=be938b6e5739470083fb26ad8a7eb628, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:00.168071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ca798-67a2-424d-8783-76dc80b7fb9b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:00.169693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ca798-67a2-424d-8783-76dc80b7fb9b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:00.183226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 386/774] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Sep 6 21:00:59 2022] GET /identity/v3/endpoints?service_id=be938b6e5739470083fb26ad8a7eb628&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:00.188963 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828558e0-8815-4f0b-8e8b-23e65cc67496 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:00.260095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:00.315832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828558e0-8815-4f0b-8e8b-23e65cc67496 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:00.350941 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9301f-eb5b-4a5c-a89f-c5c4d7ffe057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:00.351569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9301f-eb5b-4a5c-a89f-c5c4d7ffe057 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:00.351848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9301f-eb5b-4a5c-a89f-c5c4d7ffe057 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:00.352204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9301f-eb5b-4a5c-a89f-c5c4d7ffe057 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:00.352983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9301f-eb5b-4a5c-a89f-c5c4d7ffe057 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:00.354392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9301f-eb5b-4a5c-a89f-c5c4d7ffe057 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:00.360179 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 389/775] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Sep 6 21:01:00 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:00.372725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ca798-67a2-424d-8783-76dc80b7fb9b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:00.400146 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:00.406764 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70addf0c-5720-47cd-bf59-bc25e2ae301d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:00.407062 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70addf0c-5720-47cd-bf59-bc25e2ae301d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:00.407338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70addf0c-5720-47cd-bf59-bc25e2ae301d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:00.408773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 390/776] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:00.424267 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa768f4f-87f3-4a41-80ec-71db1dddfa83 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:00.424596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa768f4f-87f3-4a41-80ec-71db1dddfa83 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:00.424863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa768f4f-87f3-4a41-80ec-71db1dddfa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:00.452237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-aa768f4f-87f3-4a41-80ec-71db1dddfa83 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:00.453202 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-aa768f4f-87f3-4a41-80ec-71db1dddfa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:00.536703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 391/777] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:00.567702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ca798-67a2-424d-8783-76dc80b7fb9b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:00.608165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad82ba3-9491-49be-9aca-4a49f935c29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:00.608846 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad82ba3-9491-49be-9aca-4a49f935c29e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:00.609127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad82ba3-9491-49be-9aca-4a49f935c29e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:00.609441 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad82ba3-9491-49be-9aca-4a49f935c29e None admin] PATH_INFO: `/v3/services/network` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:00.610037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad82ba3-9491-49be-9aca-4a49f935c29e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:00.611428 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad82ba3-9491-49be-9aca-4a49f935c29e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-3ad82ba3-9491-49be-9aca-4a49f935c29e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:00.621478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:00.622485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:00.623446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 387/778] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:00 2022] GET /identity/v3/services/network => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:00.628964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa768f4f-87f3-4a41-80ec-71db1dddfa83 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:00.677971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:00.752946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa768f4f-87f3-4a41-80ec-71db1dddfa83 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:00.787392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6dfd67-6320-4a3e-98cb-a02c04515cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:00.788022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dfd67-6320-4a3e-98cb-a02c04515cce None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:00.788491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dfd67-6320-4a3e-98cb-a02c04515cce None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:00.788792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dfd67-6320-4a3e-98cb-a02c04515cce None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:00.789432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6dfd67-6320-4a3e-98cb-a02c04515cce None admin] RBAC: Adding query filter params (name=network) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:00.789827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6dfd67-6320-4a3e-98cb-a02c04515cce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:00.791108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6dfd67-6320-4a3e-98cb-a02c04515cce None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:00.796360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 392/779] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Sep 6 21:01:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:00.801976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad82ba3-9491-49be-9aca-4a49f935c29e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:00.837070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6dfd67-6320-4a3e-98cb-a02c04515cce None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:00.929000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:00.992953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad82ba3-9491-49be-9aca-4a49f935c29e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:01.012441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6dfd67-6320-4a3e-98cb-a02c04515cce None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:01.016150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:01.049464 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1916127-e10c-448e-bfd6-6ad78914b18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:01.050110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1916127-e10c-448e-bfd6-6ad78914b18b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:01.050559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1916127-e10c-448e-bfd6-6ad78914b18b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:01.050710 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1916127-e10c-448e-bfd6-6ad78914b18b None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:01.051323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1916127-e10c-448e-bfd6-6ad78914b18b None admin] RBAC: Adding query filter params (type=network) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:01.051632 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1916127-e10c-448e-bfd6-6ad78914b18b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:01.053387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1916127-e10c-448e-bfd6-6ad78914b18b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:01.061909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 388/780] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Sep 6 21:01:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:01.063222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a98c4-6c0e-41dd-963f-8efbe51ffd15 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:01.063526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a98c4-6c0e-41dd-963f-8efbe51ffd15 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:01.063814 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a98c4-6c0e-41dd-963f-8efbe51ffd15 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:01.071588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 389/781] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:01.071588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a98c4-6c0e-41dd-963f-8efbe51ffd15 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:01.078790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f9133-6a0c-4b80-95e8-e8657bed0350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:01.079706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f9133-6a0c-4b80-95e8-e8657bed0350 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:01.080164 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f9133-6a0c-4b80-95e8-e8657bed0350 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:01.080635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f9133-6a0c-4b80-95e8-e8657bed0350 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:01.092584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f9133-6a0c-4b80-95e8-e8657bed0350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-cb1f9133-6a0c-4b80-95e8-e8657bed0350 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:01.097949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:01.098891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:01.099759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:01.099759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:01.099759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:01.099759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:01.099759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:01.099759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:01.099759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:01.099759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:01.099759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:01.099759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:01.099759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:01:01.099759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:01.102612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 393/782] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:01:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:01.104134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480bbb9-7fe6-43bb-bd95-dddc8c3e90fe None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:01.104740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480bbb9-7fe6-43bb-bd95-dddc8c3e90fe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:01.105175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480bbb9-7fe6-43bb-bd95-dddc8c3e90fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:01.124807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d480bbb9-7fe6-43bb-bd95-dddc8c3e90fe None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:01.126367 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d480bbb9-7fe6-43bb-bd95-dddc8c3e90fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:01.202757 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 394/783] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:01.203562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480bbb9-7fe6-43bb-bd95-dddc8c3e90fe None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:01.220072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a98c4-6c0e-41dd-963f-8efbe51ffd15 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:01.303389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:01.311821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480bbb9-7fe6-43bb-bd95-dddc8c3e90fe None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:01.313429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f8e0a-aa80-43b3-9b4a-0b7354d9a0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:01.314067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f8e0a-aa80-43b3-9b4a-0b7354d9a0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:01.314355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f8e0a-aa80-43b3-9b4a-0b7354d9a0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:01.314630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f8e0a-aa80-43b3-9b4a-0b7354d9a0f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:01.316054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f8e0a-aa80-43b3-9b4a-0b7354d9a0f9 None admin] RBAC: Adding query filter params (interface=public, service_id=92ad3dfb471f4cf0a16fed27faf7113d, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:01.316054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f8e0a-aa80-43b3-9b4a-0b7354d9a0f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:01.317060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f8e0a-aa80-43b3-9b4a-0b7354d9a0f9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:01.327164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:01.330636 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 390/784] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Sep 6 21:01:01 2022] GET /identity/v3/endpoints?service_id=92ad3dfb471f4cf0a16fed27faf7113d&interface=public®ion_id=RegionOne => generated 194 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:01.336602 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f8e0a-aa80-43b3-9b4a-0b7354d9a0f9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:01.356215 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d8ebe-79cc-4b5c-9074-d7de77652abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:01.357293 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d8ebe-79cc-4b5c-9074-d7de77652abc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:01.357763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d8ebe-79cc-4b5c-9074-d7de77652abc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:01.358262 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d8ebe-79cc-4b5c-9074-d7de77652abc None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:01.359130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d8ebe-79cc-4b5c-9074-d7de77652abc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:01.361250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d8ebe-79cc-4b5c-9074-d7de77652abc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:01.363298 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d8ebe-79cc-4b5c-9074-d7de77652abc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:01.381431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 395/785] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:01:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:01.393648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d8ebe-79cc-4b5c-9074-d7de77652abc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:01.468792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f8e0a-aa80-43b3-9b4a-0b7354d9a0f9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:01.507279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44357c20-a4a1-4260-972b-5f7e1ea952f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:01.507915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44357c20-a4a1-4260-972b-5f7e1ea952f7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:01.508193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44357c20-a4a1-4260-972b-5f7e1ea952f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:01.508476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44357c20-a4a1-4260-972b-5f7e1ea952f7 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:01.509206 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44357c20-a4a1-4260-972b-5f7e1ea952f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:01.510665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44357c20-a4a1-4260-972b-5f7e1ea952f7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:01.517429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 391/786] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Sep 6 21:01:01 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:01.540311 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d8ebe-79cc-4b5c-9074-d7de77652abc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:01.581451 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292cc774-6332-4eec-bb0e-b9e4acb2320f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:01.582478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cc774-6332-4eec-bb0e-b9e4acb2320f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:01.582948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cc774-6332-4eec-bb0e-b9e4acb2320f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:01.583439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cc774-6332-4eec-bb0e-b9e4acb2320f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:01.597515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292cc774-6332-4eec-bb0e-b9e4acb2320f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:01.599020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292cc774-6332-4eec-bb0e-b9e4acb2320f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:01.600687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44357c20-a4a1-4260-972b-5f7e1ea952f7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-292cc774-6332-4eec-bb0e-b9e4acb2320f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:01.614989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:01.616006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:01.616945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:01.616945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:01.616945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:01.616945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 396/787] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:01.622773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292cc774-6332-4eec-bb0e-b9e4acb2320f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:01.636702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:01.755834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292cc774-6332-4eec-bb0e-b9e4acb2320f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:01.757050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:01.798464 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44357c20-a4a1-4260-972b-5f7e1ea952f7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:01.799016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba7afc-6a34-49d9-8aab-9f07e9aed908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:01.799721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba7afc-6a34-49d9-8aab-9f07e9aed908 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:01.800233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba7afc-6a34-49d9-8aab-9f07e9aed908 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:01.800575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba7afc-6a34-49d9-8aab-9f07e9aed908 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:01.802153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba7afc-6a34-49d9-8aab-9f07e9aed908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:01.802530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba7afc-6a34-49d9-8aab-9f07e9aed908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:01.803991 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba7afc-6a34-49d9-8aab-9f07e9aed908 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:01.824841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 397/788] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:01.827243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba7afc-6a34-49d9-8aab-9f07e9aed908 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:01.849780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53d6cf-399b-4553-a228-7b5dba835cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:01.850418 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53d6cf-399b-4553-a228-7b5dba835cdd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:01.850690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53d6cf-399b-4553-a228-7b5dba835cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:01.850954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53d6cf-399b-4553-a228-7b5dba835cdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:01.862464 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53d6cf-399b-4553-a228-7b5dba835cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6d53d6cf-399b-4553-a228-7b5dba835cdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:01.868313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:01.869307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:01.870138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:01.870138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:01.870138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:01.870138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:01.870138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:01.870138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:01.870138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:01.870138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:01.870138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:01.870138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:01.870138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:01:01.870138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:01.870138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 392/789] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:01:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:01.876997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53d6cf-399b-4553-a228-7b5dba835cdd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:01.917983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:01.956119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba7afc-6a34-49d9-8aab-9f07e9aed908 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:01.992027 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7559370-bdd0-45a3-9b82-2c53931892cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:01.993304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7559370-bdd0-45a3-9b82-2c53931892cb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:01.993928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7559370-bdd0-45a3-9b82-2c53931892cb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:01.994398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7559370-bdd0-45a3-9b82-2c53931892cb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:02.002206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7559370-bdd0-45a3-9b82-2c53931892cb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:02.003959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7559370-bdd0-45a3-9b82-2c53931892cb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f7559370-bdd0-45a3-9b82-2c53931892cb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:01:02.016927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:02.018313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:02.019244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:02.019244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:02.019244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:01:02.019244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:01:02.019244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:01:02.019244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:01:02.019244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:01:02.019244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:01:02.019244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 21:01:02.019244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:02.021811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 398/790] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:01:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:02.027861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7559370-bdd0-45a3-9b82-2c53931892cb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:02.055691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53d6cf-399b-4553-a228-7b5dba835cdd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:02.103544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:02.132219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00332ea-8454-4f1e-8dd0-ac0615e56938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:02.133110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00332ea-8454-4f1e-8dd0-ac0615e56938 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:02.133294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00332ea-8454-4f1e-8dd0-ac0615e56938 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:02.133542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00332ea-8454-4f1e-8dd0-ac0615e56938 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:02.134549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00332ea-8454-4f1e-8dd0-ac0615e56938 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:02.134776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00332ea-8454-4f1e-8dd0-ac0615e56938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:02.136195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00332ea-8454-4f1e-8dd0-ac0615e56938 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:02.144511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7559370-bdd0-45a3-9b82-2c53931892cb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:02.155342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 393/791] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:01:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:02.160520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00332ea-8454-4f1e-8dd0-ac0615e56938 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:02.185478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:02.204161 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ff26a-6c06-410d-9d3e-258224c5b615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:02.209376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ff26a-6c06-410d-9d3e-258224c5b615 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:02.209376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ff26a-6c06-410d-9d3e-258224c5b615 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:02.209376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ff26a-6c06-410d-9d3e-258224c5b615 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:02.209376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ff26a-6c06-410d-9d3e-258224c5b615 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:02.209376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ff26a-6c06-410d-9d3e-258224c5b615 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:02.209376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ff26a-6c06-410d-9d3e-258224c5b615 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:02.235052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 399/792] 10.222.0.79 () {56 vars in 1135 bytes} [Tue Sep 6 21:01:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:02.240936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ff26a-6c06-410d-9d3e-258224c5b615 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:02.311466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00332ea-8454-4f1e-8dd0-ac0615e56938 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:02.359804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13972583-cf21-41b7-b3a2-cdb7101c3662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:02.361015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13972583-cf21-41b7-b3a2-cdb7101c3662 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:02.361493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13972583-cf21-41b7-b3a2-cdb7101c3662 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:02.361906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13972583-cf21-41b7-b3a2-cdb7101c3662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:02.362373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ff26a-6c06-410d-9d3e-258224c5b615 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:02.379457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13972583-cf21-41b7-b3a2-cdb7101c3662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:02.381742 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13972583-cf21-41b7-b3a2-cdb7101c3662 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:02.402062 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9760a9-3a33-450c-96f6-b23b6b1761af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:02.402810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9760a9-3a33-450c-96f6-b23b6b1761af None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:02.403088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9760a9-3a33-450c-96f6-b23b6b1761af None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:02.403353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9760a9-3a33-450c-96f6-b23b6b1761af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-13972583-cf21-41b7-b3a2-cdb7101c3662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:02.403626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:02.404751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:02.405795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:02.405795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:02.405795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:02.406401 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 394/793] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:02.407598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95711c10-0a4a-424b-a086-3967a6644531 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:02.407873 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95711c10-0a4a-424b-a086-3967a6644531 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:02.408132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95711c10-0a4a-424b-a086-3967a6644531 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:02.409631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 395/794] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:02.415370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883a798-19d5-41cb-b94b-1b6902c42c46 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:02.415648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883a798-19d5-41cb-b94b-1b6902c42c46 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:02.415916 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883a798-19d5-41cb-b94b-1b6902c42c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:02.420800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9760a9-3a33-450c-96f6-b23b6b1761af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:02.422748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9760a9-3a33-450c-96f6-b23b6b1761af None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:02.434824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6883a798-19d5-41cb-b94b-1b6902c42c46 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:02.436533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6883a798-19d5-41cb-b94b-1b6902c42c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8f9760a9-3a33-450c-96f6-b23b6b1761af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:02.439423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:02.440517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:02.441646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:02.441646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:02.441646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:02.447686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 400/795] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:02.449072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9760a9-3a33-450c-96f6-b23b6b1761af None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:02.471414 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:02.495701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:02.525841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 396/796] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:02.526658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883a798-19d5-41cb-b94b-1b6902c42c46 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:02.620833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9760a9-3a33-450c-96f6-b23b6b1761af None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:02.637297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883a798-19d5-41cb-b94b-1b6902c42c46 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:02.672731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c56b5-4dd9-4cb9-8e74-9cb0624d7861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:02.673350 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c56b5-4dd9-4cb9-8e74-9cb0624d7861 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:02.673622 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c56b5-4dd9-4cb9-8e74-9cb0624d7861 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:02.673888 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c56b5-4dd9-4cb9-8e74-9cb0624d7861 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:02.674703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c56b5-4dd9-4cb9-8e74-9cb0624d7861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:02.674999 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c56b5-4dd9-4cb9-8e74-9cb0624d7861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:02.676373 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c56b5-4dd9-4cb9-8e74-9cb0624d7861 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:02.689241 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 397/797] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:02.694766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c56b5-4dd9-4cb9-8e74-9cb0624d7861 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:02.708036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd593e0c-fb3a-40b3-ba03-00f8ca3ff119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:02.708680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd593e0c-fb3a-40b3-ba03-00f8ca3ff119 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:02.708962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd593e0c-fb3a-40b3-ba03-00f8ca3ff119 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:02.709231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd593e0c-fb3a-40b3-ba03-00f8ca3ff119 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:02.709873 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd593e0c-fb3a-40b3-ba03-00f8ca3ff119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:02.710169 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd593e0c-fb3a-40b3-ba03-00f8ca3ff119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:02.711768 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd593e0c-fb3a-40b3-ba03-00f8ca3ff119 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:02.750913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 401/798] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:02.752220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:02.757297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd593e0c-fb3a-40b3-ba03-00f8ca3ff119 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:02.800224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c56b5-4dd9-4cb9-8e74-9cb0624d7861 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:02.842621 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b52e23-7ec1-46d3-a7fc-ba5f806007c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:02.843523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b52e23-7ec1-46d3-a7fc-ba5f806007c0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:02.843938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b52e23-7ec1-46d3-a7fc-ba5f806007c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:02.844371 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b52e23-7ec1-46d3-a7fc-ba5f806007c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:02.852324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b52e23-7ec1-46d3-a7fc-ba5f806007c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:02.854052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b52e23-7ec1-46d3-a7fc-ba5f806007c0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a4b52e23-7ec1-46d3-a7fc-ba5f806007c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:01:02.866832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:02.867760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:02.868641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:01:02.868641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:01:02.868641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:01:02.868641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:01:02.868641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:01:02.868641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:01:02.868641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:02.868641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:02.871555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 398/799] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:02.876753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b52e23-7ec1-46d3-a7fc-ba5f806007c0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:02.902222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:02.906758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd593e0c-fb3a-40b3-ba03-00f8ca3ff119 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:02.945391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0814be-355b-4046-b4fc-39f65af5e03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:02.946144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0814be-355b-4046-b4fc-39f65af5e03c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:02.946497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0814be-355b-4046-b4fc-39f65af5e03c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:02.946824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0814be-355b-4046-b4fc-39f65af5e03c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:02.955084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0814be-355b-4046-b4fc-39f65af5e03c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:02.956772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0814be-355b-4046-b4fc-39f65af5e03c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7f0814be-355b-4046-b4fc-39f65af5e03c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:01:02.969615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:02.970585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:02.971487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:02.971487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:02.971487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:01:02.971487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:01:02.971487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:01:02.971487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:01:02.971487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:01:02.971487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:01:02.971487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 21:01:02.971487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:02.971487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 402/800] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Sep 6 21:01:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:02.972066 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0814be-355b-4046-b4fc-39f65af5e03c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:02.984014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:02.998421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b52e23-7ec1-46d3-a7fc-ba5f806007c0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:03.038573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03e797-7600-452f-a0d9-50878c91ac53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:03.039217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03e797-7600-452f-a0d9-50878c91ac53 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:03.039490 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03e797-7600-452f-a0d9-50878c91ac53 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:03.039880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03e797-7600-452f-a0d9-50878c91ac53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:03.050237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03e797-7600-452f-a0d9-50878c91ac53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-bb03e797-7600-452f-a0d9-50878c91ac53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:03.055476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:03.057102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:03.057957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:03.057957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:03.057957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:03.057957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:03.057957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:03.057957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:03.057957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:03.057957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:03.057957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:03.057957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:03.057957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:01:03.057957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:03.059739 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 399/801] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:01:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:03.061605 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03e797-7600-452f-a0d9-50878c91ac53 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:03.085443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0814be-355b-4046-b4fc-39f65af5e03c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:03.120625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2c292-a558-45b4-8a92-e3d5df3813aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:03.121176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2c292-a558-45b4-8a92-e3d5df3813aa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:03.121509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2c292-a558-45b4-8a92-e3d5df3813aa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:03.121771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2c292-a558-45b4-8a92-e3d5df3813aa None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:03.122569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2c292-a558-45b4-8a92-e3d5df3813aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:03.122764 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2c292-a558-45b4-8a92-e3d5df3813aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:03.124222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2c292-a558-45b4-8a92-e3d5df3813aa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:03.137488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 403/802] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:01:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:03.143391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd81dfa-72e4-4ec1-b2be-b7bde4846984 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:03.143700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd81dfa-72e4-4ec1-b2be-b7bde4846984 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:03.144131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd81dfa-72e4-4ec1-b2be-b7bde4846984 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:03.145395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 404/803] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:03.145920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd81dfa-72e4-4ec1-b2be-b7bde4846984 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:03.167116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:03.170591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03e797-7600-452f-a0d9-50878c91ac53 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:03.207299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79473f7-b7ca-44d2-b129-9ac3c26fb7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:03.208040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79473f7-b7ca-44d2-b129-9ac3c26fb7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:03.208321 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79473f7-b7ca-44d2-b129-9ac3c26fb7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:03.208594 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79473f7-b7ca-44d2-b129-9ac3c26fb7b1 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:03.209376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79473f7-b7ca-44d2-b129-9ac3c26fb7b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:03.209676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79473f7-b7ca-44d2-b129-9ac3c26fb7b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:03.211108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79473f7-b7ca-44d2-b129-9ac3c26fb7b1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:03.234230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 400/804] 10.222.0.79 () {56 vars in 1149 bytes} [Tue Sep 6 21:01:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:03.234230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79473f7-b7ca-44d2-b129-9ac3c26fb7b1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:03.240959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:03.269957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd81dfa-72e4-4ec1-b2be-b7bde4846984 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:03.305593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e962c7-08cb-4792-ae72-632c51473d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:03.306210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e962c7-08cb-4792-ae72-632c51473d3c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:03.306481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e962c7-08cb-4792-ae72-632c51473d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:03.306745 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e962c7-08cb-4792-ae72-632c51473d3c None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:03.307447 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e962c7-08cb-4792-ae72-632c51473d3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:03.307778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e962c7-08cb-4792-ae72-632c51473d3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:03.309314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e962c7-08cb-4792-ae72-632c51473d3c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:03.318542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 405/805] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:01:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:03.319536 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4730975-95dd-4e91-af4b-45d9e0455cba None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:03.319817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4730975-95dd-4e91-af4b-45d9e0455cba None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:03.320087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4730975-95dd-4e91-af4b-45d9e0455cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:03.338290 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d4730975-95dd-4e91-af4b-45d9e0455cba None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:03.339223 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d4730975-95dd-4e91-af4b-45d9e0455cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:03.342637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79473f7-b7ca-44d2-b129-9ac3c26fb7b1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:03.378251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895002b2-db1d-4f93-9d11-fd142a6cfa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:03.378835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895002b2-db1d-4f93-9d11-fd142a6cfa76 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:03.379111 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895002b2-db1d-4f93-9d11-fd142a6cfa76 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:03.379380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895002b2-db1d-4f93-9d11-fd142a6cfa76 None admin] PATH_INFO: `/v3/projects/05af249ba0a047fe83aefb078e203e53/users/061c584de8264868af8782402a98a72e/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:03.381273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895002b2-db1d-4f93-9d11-fd142a6cfa76 None admin] RBAC: Authorizing `identity:create_grant(project_id=05af249ba0a047fe83aefb078e203e53, user_id=061c584de8264868af8782402a98a72e, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:03.382705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895002b2-db1d-4f93-9d11-fd142a6cfa76 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:03.396788 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 401/806] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:01:03 2022] PUT /identity/v3/projects/05af249ba0a047fe83aefb078e203e53/users/061c584de8264868af8782402a98a72e/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:01:03.397867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895002b2-db1d-4f93-9d11-fd142a6cfa76 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:03.425220 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 406/807] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:03.426458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4730975-95dd-4e91-af4b-45d9e0455cba None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:03.454421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:03.517145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:03.606673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895002b2-db1d-4f93-9d11-fd142a6cfa76 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:03.609248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4730975-95dd-4e91-af4b-45d9e0455cba None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:03.647266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cdc959-2c67-4463-a22b-ca651ffa5a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:03.647920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdc959-2c67-4463-a22b-ca651ffa5a7c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:03.648114 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdc959-2c67-4463-a22b-ca651ffa5a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:03.648392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdc959-2c67-4463-a22b-ca651ffa5a7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:03.653320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bb4d1-e011-4c5d-b13a-8e4045dd6fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:03.654531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bb4d1-e011-4c5d-b13a-8e4045dd6fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:03.654996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bb4d1-e011-4c5d-b13a-8e4045dd6fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:03.655542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bb4d1-e011-4c5d-b13a-8e4045dd6fd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:03.659507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdc959-2c67-4463-a22b-ca651ffa5a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:03.661021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdc959-2c67-4463-a22b-ca651ffa5a7c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:03.669158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bb4d1-e011-4c5d-b13a-8e4045dd6fd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:03.671034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bb4d1-e011-4c5d-b13a-8e4045dd6fd3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-39cdc959-2c67-4463-a22b-ca651ffa5a7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:03.676256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:03.677383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:03.678285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:03.678285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:03.678285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:03.682917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 407/808] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:03.684323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cdc959-2c67-4463-a22b-ca651ffa5a7c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ec7bb4d1-e011-4c5d-b13a-8e4045dd6fd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:03.690290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:03.691340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:03.692220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:03.692220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:03.692220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:03.694338 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 402/809] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:03.699910 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bb4d1-e011-4c5d-b13a-8e4045dd6fd3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:03.750221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:03.792866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cdc959-2c67-4463-a22b-ca651ffa5a7c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:03.816487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bb4d1-e011-4c5d-b13a-8e4045dd6fd3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:03.828736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac62e87-473e-4bd5-987a-997f4380b60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:03.829360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac62e87-473e-4bd5-987a-997f4380b60a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:03.829641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac62e87-473e-4bd5-987a-997f4380b60a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:03.829912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac62e87-473e-4bd5-987a-997f4380b60a None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:03.830568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac62e87-473e-4bd5-987a-997f4380b60a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:03.830872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac62e87-473e-4bd5-987a-997f4380b60a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:03.832266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac62e87-473e-4bd5-987a-997f4380b60a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:03.835457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:03.850882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 408/810] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:03.852038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac62e87-473e-4bd5-987a-997f4380b60a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:03.854852 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50956696-8695-44c4-a6eb-fb22ea16ce0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:03.855579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50956696-8695-44c4-a6eb-fb22ea16ce0e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:03.855937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50956696-8695-44c4-a6eb-fb22ea16ce0e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:03.856272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50956696-8695-44c4-a6eb-fb22ea16ce0e None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:03.856964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50956696-8695-44c4-a6eb-fb22ea16ce0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:03.857268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50956696-8695-44c4-a6eb-fb22ea16ce0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:03.858651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50956696-8695-44c4-a6eb-fb22ea16ce0e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:03.884753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 403/811] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:03.885759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50956696-8695-44c4-a6eb-fb22ea16ce0e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:03.960074 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac62e87-473e-4bd5-987a-997f4380b60a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:03.985165 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:03.995659 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50956696-8695-44c4-a6eb-fb22ea16ce0e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:03.999334 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4f880-e10f-41ad-98ef-d954ff2138f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:04.000068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4f880-e10f-41ad-98ef-d954ff2138f6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:04.000416 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4f880-e10f-41ad-98ef-d954ff2138f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:04.000805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4f880-e10f-41ad-98ef-d954ff2138f6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:04.001471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4f880-e10f-41ad-98ef-d954ff2138f6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:04.002970 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4f880-e10f-41ad-98ef-d954ff2138f6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-58e4f880-e10f-41ad-98ef-d954ff2138f6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:04.013243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:04.014300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:04.015123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 409/812] 10.222.0.79 () {56 vars in 1129 bytes} [Tue Sep 6 21:01:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:04.016738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4f880-e10f-41ad-98ef-d954ff2138f6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:04.041618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8236e-d400-49bd-b549-0039f7cc0962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:04.042330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8236e-d400-49bd-b549-0039f7cc0962 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:04.042607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8236e-d400-49bd-b549-0039f7cc0962 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:04.042920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8236e-d400-49bd-b549-0039f7cc0962 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:04.052393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8236e-d400-49bd-b549-0039f7cc0962 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:04.054415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8236e-d400-49bd-b549-0039f7cc0962 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-67c8236e-d400-49bd-b549-0039f7cc0962 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:01:04.068388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:04.070335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:04.073000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:04.073000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:04.073000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:01:04.073000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:01:04.073000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:01:04.073000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:01:04.073000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:01:04.073000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:01:04.073000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:01:04.073000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:04.080691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 404/813] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Sep 6 21:01:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:04.080691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8236e-d400-49bd-b549-0039f7cc0962 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:04.129529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:04.137587 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4f880-e10f-41ad-98ef-d954ff2138f6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:04.176385 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaaec30-f329-41d1-b2b6-314434a60c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:04.177080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaec30-f329-41d1-b2b6-314434a60c97 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:04.177358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaec30-f329-41d1-b2b6-314434a60c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:04.177636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaec30-f329-41d1-b2b6-314434a60c97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:04.185745 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaaec30-f329-41d1-b2b6-314434a60c97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:04.187341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaaec30-f329-41d1-b2b6-314434a60c97 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ffaaec30-f329-41d1-b2b6-314434a60c97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:01:04.200024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:04.201065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:04.201948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:01:04.201948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:01:04.201948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:01:04.201948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:01:04.201948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:01:04.201948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:01:04.201948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:04.201948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:04.206118 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 410/814] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:04.207918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaaec30-f329-41d1-b2b6-314434a60c97 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:04.210316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8236e-d400-49bd-b549-0039f7cc0962 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:04.263371 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cca910-5d23-4555-9c8d-5db71d3851cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:04.264008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cca910-5d23-4555-9c8d-5db71d3851cb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:04.264314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cca910-5d23-4555-9c8d-5db71d3851cb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:04.264703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cca910-5d23-4555-9c8d-5db71d3851cb None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:04.265842 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cca910-5d23-4555-9c8d-5db71d3851cb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:04.266334 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cca910-5d23-4555-9c8d-5db71d3851cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:04.268261 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cca910-5d23-4555-9c8d-5db71d3851cb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:04.271222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:04.274954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 405/815] 10.222.0.79 () {56 vars in 1124 bytes} [Tue Sep 6 21:01:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:04.276332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f21d5-c4cb-4745-bfec-d5becbb08135 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:04.276616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f21d5-c4cb-4745-bfec-d5becbb08135 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:04.277107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f21d5-c4cb-4745-bfec-d5becbb08135 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:04.280740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 406/816] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:04.280740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f21d5-c4cb-4745-bfec-d5becbb08135 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:04.356155 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaaec30-f329-41d1-b2b6-314434a60c97 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:04.396691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec692c3a-a513-4970-b23e-10b9bdb92ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:04.397649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec692c3a-a513-4970-b23e-10b9bdb92ece None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:04.398128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec692c3a-a513-4970-b23e-10b9bdb92ece None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:04.398575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec692c3a-a513-4970-b23e-10b9bdb92ece None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:04.399506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec692c3a-a513-4970-b23e-10b9bdb92ece None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:04.399852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec692c3a-a513-4970-b23e-10b9bdb92ece None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:04.401351 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec692c3a-a513-4970-b23e-10b9bdb92ece None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:04.408857 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f21d5-c4cb-4745-bfec-d5becbb08135 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:04.411332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:04.425734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 411/817] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Sep 6 21:01:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:04.426639 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec692c3a-a513-4970-b23e-10b9bdb92ece None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:04.463740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b860f0-5029-4b27-99a1-31230c1b65d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:04.464263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b860f0-5029-4b27-99a1-31230c1b65d2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:04.464537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b860f0-5029-4b27-99a1-31230c1b65d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:04.464842 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b860f0-5029-4b27-99a1-31230c1b65d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:04.465604 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b860f0-5029-4b27-99a1-31230c1b65d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:04.465904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b860f0-5029-4b27-99a1-31230c1b65d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:04.467353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b860f0-5029-4b27-99a1-31230c1b65d2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:04.481336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 407/818] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:01:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:04.483031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c3a95-e60b-4ad7-8588-98028e059b26 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:04.483507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c3a95-e60b-4ad7-8588-98028e059b26 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:04.483966 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c3a95-e60b-4ad7-8588-98028e059b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:04.504687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-733c3a95-e60b-4ad7-8588-98028e059b26 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:04.505882 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-733c3a95-e60b-4ad7-8588-98028e059b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:04.588972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec692c3a-a513-4970-b23e-10b9bdb92ece None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:04.593714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:04.608692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 408/819] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:04.608692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c3a95-e60b-4ad7-8588-98028e059b26 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:04.641115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f355db-7ddd-492a-bb48-c69c95e7e7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:04.641806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f355db-7ddd-492a-bb48-c69c95e7e7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:04.642082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f355db-7ddd-492a-bb48-c69c95e7e7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:04.642348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f355db-7ddd-492a-bb48-c69c95e7e7c0 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:04.643678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f355db-7ddd-492a-bb48-c69c95e7e7c0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:04.643996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f355db-7ddd-492a-bb48-c69c95e7e7c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:04.646480 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f355db-7ddd-492a-bb48-c69c95e7e7c0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:04.655667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 412/820] 10.222.0.79 () {56 vars in 1124 bytes} [Tue Sep 6 21:01:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:04.658765 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f355db-7ddd-492a-bb48-c69c95e7e7c0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:04.739568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:04.750847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c3a95-e60b-4ad7-8588-98028e059b26 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:04.796484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f355db-7ddd-492a-bb48-c69c95e7e7c0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:04.796877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a57020-a4a0-43ff-a2c6-8e8a2dd35e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:04.797836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a57020-a4a0-43ff-a2c6-8e8a2dd35e83 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:04.798178 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a57020-a4a0-43ff-a2c6-8e8a2dd35e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:04.798511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a57020-a4a0-43ff-a2c6-8e8a2dd35e83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:04.843856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f80826-d100-44fc-b8f3-ec8fe90ac15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:04.844877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f80826-d100-44fc-b8f3-ec8fe90ac15f None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:04.845419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f80826-d100-44fc-b8f3-ec8fe90ac15f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:04.845860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f80826-d100-44fc-b8f3-ec8fe90ac15f None admin] PATH_INFO: `/v3/projects/05af249ba0a047fe83aefb078e203e53/users/c09b4900d97b4b35b617af9a42baf9ee/roles/b3f49281e89f4fa1a2d119e2fb4922e5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:04.865166 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f80826-d100-44fc-b8f3-ec8fe90ac15f None admin] RBAC: Authorizing `identity:create_grant(project_id=05af249ba0a047fe83aefb078e203e53, user_id=c09b4900d97b4b35b617af9a42baf9ee, role_id=b3f49281e89f4fa1a2d119e2fb4922e5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:04.866981 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f80826-d100-44fc-b8f3-ec8fe90ac15f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:04.884998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a57020-a4a0-43ff-a2c6-8e8a2dd35e83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:04.886315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 413/821] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:01:04 2022] PUT /identity/v3/projects/05af249ba0a047fe83aefb078e203e53/users/c09b4900d97b4b35b617af9a42baf9ee/roles/b3f49281e89f4fa1a2d119e2fb4922e5 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:01:04.886511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a57020-a4a0-43ff-a2c6-8e8a2dd35e83 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:04.897681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f80826-d100-44fc-b8f3-ec8fe90ac15f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:04.934147 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:05.002640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:05.014198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 409/822] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:01:04 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:01:05.015087 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a57020-a4a0-43ff-a2c6-8e8a2dd35e83 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:05.061978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f80826-d100-44fc-b8f3-ec8fe90ac15f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:05.102200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de36ba-8258-433b-83b3-6eb741b4f517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:05.102989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de36ba-8258-433b-83b3-6eb741b4f517 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:05.103272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de36ba-8258-433b-83b3-6eb741b4f517 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:05.103553 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de36ba-8258-433b-83b3-6eb741b4f517 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:05.104221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de36ba-8258-433b-83b3-6eb741b4f517 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:05.105656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de36ba-8258-433b-83b3-6eb741b4f517 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:05.131664 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 414/823] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:01:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:05.132500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de36ba-8258-433b-83b3-6eb741b4f517 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:05.145722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a57020-a4a0-43ff-a2c6-8e8a2dd35e83 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:05.191045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa49b2c-c5e8-400e-a9ac-bb4ac8e16fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:05.192066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa49b2c-c5e8-400e-a9ac-bb4ac8e16fbf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:05.192540 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa49b2c-c5e8-400e-a9ac-bb4ac8e16fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:05.193052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa49b2c-c5e8-400e-a9ac-bb4ac8e16fbf None admin] PATH_INFO: `/v3/services/network` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:05.193860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa49b2c-c5e8-400e-a9ac-bb4ac8e16fbf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:05.195465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa49b2c-c5e8-400e-a9ac-bb4ac8e16fbf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0aa49b2c-c5e8-400e-a9ac-bb4ac8e16fbf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:05.205721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:05.206779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 21:01:05.207596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:05.210554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 410/824] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:05 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:05.222398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa49b2c-c5e8-400e-a9ac-bb4ac8e16fbf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:05.331361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:05.391353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa49b2c-c5e8-400e-a9ac-bb4ac8e16fbf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:05.400822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:05.428184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de36ba-8258-433b-83b3-6eb741b4f517 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:05.454211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8594d8-f7f6-4f14-adfa-780163abd589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:05.455198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8594d8-f7f6-4f14-adfa-780163abd589 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:05.455673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8594d8-f7f6-4f14-adfa-780163abd589 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:05.456165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8594d8-f7f6-4f14-adfa-780163abd589 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:05.457601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8594d8-f7f6-4f14-adfa-780163abd589 None admin] RBAC: Adding query filter params (name=network) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:05.458108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8594d8-f7f6-4f14-adfa-780163abd589 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:05.460212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8594d8-f7f6-4f14-adfa-780163abd589 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:05.469918 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 411/825] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Sep 6 21:01:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:05.477406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8594d8-f7f6-4f14-adfa-780163abd589 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:05.482110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef11b8-644a-4663-88c3-e8f107b3dd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:05.483095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef11b8-644a-4663-88c3-e8f107b3dd45 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:05.483566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef11b8-644a-4663-88c3-e8f107b3dd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:05.484026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef11b8-644a-4663-88c3-e8f107b3dd45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:05.495701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef11b8-644a-4663-88c3-e8f107b3dd45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:05.497678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef11b8-644a-4663-88c3-e8f107b3dd45 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-55ef11b8-644a-4663-88c3-e8f107b3dd45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:05.514722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:05.515706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:05.516588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:05.516588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:05.516588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:05.518812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 415/826] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:05.523994 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef11b8-644a-4663-88c3-e8f107b3dd45 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:05.596679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8594d8-f7f6-4f14-adfa-780163abd589 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:05.608475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:05.627523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b77505-ab85-4403-a9d6-745d538e3ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:05.628153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b77505-ab85-4403-a9d6-745d538e3ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:05.628430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b77505-ab85-4403-a9d6-745d538e3ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:05.628795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b77505-ab85-4403-a9d6-745d538e3ff8 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:05.629524 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b77505-ab85-4403-a9d6-745d538e3ff8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:05.629728 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b77505-ab85-4403-a9d6-745d538e3ff8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:05.631111 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b77505-ab85-4403-a9d6-745d538e3ff8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:05.640441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef11b8-644a-4663-88c3-e8f107b3dd45 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:05.640817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 412/827] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Sep 6 21:01:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:05.642763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b77505-ab85-4403-a9d6-745d538e3ff8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:05.670171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:05.682611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04078c7d-c7de-4d62-8741-6ddfe69470a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:05.683597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04078c7d-c7de-4d62-8741-6ddfe69470a8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:05.684104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04078c7d-c7de-4d62-8741-6ddfe69470a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:05.684562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04078c7d-c7de-4d62-8741-6ddfe69470a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:05.685924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04078c7d-c7de-4d62-8741-6ddfe69470a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:05.686434 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04078c7d-c7de-4d62-8741-6ddfe69470a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:05.688112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04078c7d-c7de-4d62-8741-6ddfe69470a8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:05.701395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 416/828] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:05.708797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04078c7d-c7de-4d62-8741-6ddfe69470a8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:05.791389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b77505-ab85-4403-a9d6-745d538e3ff8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:05.854244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0543921-58f3-4cf1-b826-92232d913284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:05.854784 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0543921-58f3-4cf1-b826-92232d913284 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:05.855063 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0543921-58f3-4cf1-b826-92232d913284 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:05.855336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0543921-58f3-4cf1-b826-92232d913284 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:05.856094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0543921-58f3-4cf1-b826-92232d913284 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:05.857523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0543921-58f3-4cf1-b826-92232d913284 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:05.886829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04078c7d-c7de-4d62-8741-6ddfe69470a8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:05.889028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 413/829] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:01:05 2022] POST /identity/v3/endpoints => generated 337 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:05.890476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6d5ab-94ee-4b3c-9566-afe602d58c19 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:05.890765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6d5ab-94ee-4b3c-9566-afe602d58c19 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:05.891042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6d5ab-94ee-4b3c-9566-afe602d58c19 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:05.897912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 414/830] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:05.899467 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdbdbc-b2dc-49be-b031-875f72301fe4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:05.899755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdbdbc-b2dc-49be-b031-875f72301fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:05.900022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdbdbc-b2dc-49be-b031-875f72301fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:05.928113 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8bfdbdbc-b2dc-49be-b031-875f72301fe4 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:05.929621 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8bfdbdbc-b2dc-49be-b031-875f72301fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:05.961112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09393f9a-5430-43b0-b1b3-8f30ebec004d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:05.961767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09393f9a-5430-43b0-b1b3-8f30ebec004d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:05.962045 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09393f9a-5430-43b0-b1b3-8f30ebec004d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:05.962317 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09393f9a-5430-43b0-b1b3-8f30ebec004d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:05.970049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09393f9a-5430-43b0-b1b3-8f30ebec004d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:05.971583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09393f9a-5430-43b0-b1b3-8f30ebec004d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:05.972432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-09393f9a-5430-43b0-b1b3-8f30ebec004d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:01:05.984100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:05.985138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:05.986026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:01:05.986026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:01:05.986026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:01:05.986026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:01:05.986026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:01:05.986026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:01:05.986026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:05.986026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:05.986026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 417/831] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:05.992929 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09393f9a-5430-43b0-b1b3-8f30ebec004d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:06.047669 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 415/832] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:06.090962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:06.142234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09393f9a-5430-43b0-b1b3-8f30ebec004d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:06.183078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bd1fc-4d60-4da6-825f-4852bd0b6562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:06.183700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bd1fc-4d60-4da6-825f-4852bd0b6562 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:06.184043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bd1fc-4d60-4da6-825f-4852bd0b6562 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:06.184314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bd1fc-4d60-4da6-825f-4852bd0b6562 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:06.185396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bd1fc-4d60-4da6-825f-4852bd0b6562 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:06.185698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bd1fc-4d60-4da6-825f-4852bd0b6562 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:06.187158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bd1fc-4d60-4da6-825f-4852bd0b6562 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:06.201805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 418/833] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:01:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:06.208020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdbdbc-b2dc-49be-b031-875f72301fe4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:06.314009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdbdbc-b2dc-49be-b031-875f72301fe4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:06.356957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190316fb-280b-4208-b166-41a7c063ab76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:06.357570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190316fb-280b-4208-b166-41a7c063ab76 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:06.357900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190316fb-280b-4208-b166-41a7c063ab76 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:06.358177 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190316fb-280b-4208-b166-41a7c063ab76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:06.358908 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190316fb-280b-4208-b166-41a7c063ab76 None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=12d51454d20c46d7a5e6d963a2f987cc) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:06.359215 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190316fb-280b-4208-b166-41a7c063ab76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:06.360674 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190316fb-280b-4208-b166-41a7c063ab76 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:06.364970 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bd1fc-4d60-4da6-825f-4852bd0b6562 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:06.371132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 416/834] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:01:06 2022] GET /identity/v3/role_assignments?user.id=12d51454d20c46d7a5e6d963a2f987cc&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=3b30504991194002b41f8587d0851c10 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:06.503878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bd1fc-4d60-4da6-825f-4852bd0b6562 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:06.505651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:06.545449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6571fac-9096-40bc-8055-7be6fc7f4b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:06.546095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6571fac-9096-40bc-8055-7be6fc7f4b06 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:06.546370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6571fac-9096-40bc-8055-7be6fc7f4b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:06.546637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6571fac-9096-40bc-8055-7be6fc7f4b06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:06.562543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6571fac-9096-40bc-8055-7be6fc7f4b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c6571fac-9096-40bc-8055-7be6fc7f4b06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:06.567593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:06.569778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:06.571869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:06.571869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:06.571869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:06.571869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:06.571869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:06.571869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:06.571869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:06.571869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:06.571869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:06.571869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:06.571869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:01:06.571869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:06.571869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 419/835] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:01:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:06.575193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190316fb-280b-4208-b166-41a7c063ab76 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:06.626933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:06.693344 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190316fb-280b-4208-b166-41a7c063ab76 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:06.728568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a388706-da90-4a37-8640-e21455f59010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:06.729218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a388706-da90-4a37-8640-e21455f59010 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:06.729500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a388706-da90-4a37-8640-e21455f59010 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:06.729777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a388706-da90-4a37-8640-e21455f59010 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:06.730473 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a388706-da90-4a37-8640-e21455f59010 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:06.730774 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a388706-da90-4a37-8640-e21455f59010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:06.732238 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a388706-da90-4a37-8640-e21455f59010 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:06.741439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 417/836] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:01:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:06.747275 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6571fac-9096-40bc-8055-7be6fc7f4b06 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:06.847983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6571fac-9096-40bc-8055-7be6fc7f4b06 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:06.882990 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f5e4c-7e38-4f59-b9db-007232eca864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:06.884192 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f5e4c-7e38-4f59-b9db-007232eca864 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:06.884747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f5e4c-7e38-4f59-b9db-007232eca864 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:06.885253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f5e4c-7e38-4f59-b9db-007232eca864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:06.897946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f5e4c-7e38-4f59-b9db-007232eca864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:06.899701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f5e4c-7e38-4f59-b9db-007232eca864 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ea2f5e4c-7e38-4f59-b9db-007232eca864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:06.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:06.916777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:06.917688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:06.917688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:06.917688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:06.918412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 420/837] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:06.924562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a388706-da90-4a37-8640-e21455f59010 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:07.044167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a388706-da90-4a37-8640-e21455f59010 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:07.045865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:07.081257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20698370-8d9b-48fa-9fc5-85bc26671e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:07.081877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20698370-8d9b-48fa-9fc5-85bc26671e01 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:07.082156 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20698370-8d9b-48fa-9fc5-85bc26671e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:07.082423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20698370-8d9b-48fa-9fc5-85bc26671e01 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:07.083079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20698370-8d9b-48fa-9fc5-85bc26671e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:07.083380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20698370-8d9b-48fa-9fc5-85bc26671e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:07.084859 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20698370-8d9b-48fa-9fc5-85bc26671e01 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:07.098790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 418/838] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:07.104395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f5e4c-7e38-4f59-b9db-007232eca864 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:07.120874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:07.212613 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f5e4c-7e38-4f59-b9db-007232eca864 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:07.247620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26209e4b-62ea-4f93-88c0-4ad8136f3334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:07.248271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26209e4b-62ea-4f93-88c0-4ad8136f3334 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:07.248582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26209e4b-62ea-4f93-88c0-4ad8136f3334 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:07.248925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26209e4b-62ea-4f93-88c0-4ad8136f3334 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:07.257096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26209e4b-62ea-4f93-88c0-4ad8136f3334 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:07.258578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26209e4b-62ea-4f93-88c0-4ad8136f3334 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-26209e4b-62ea-4f93-88c0-4ad8136f3334 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:01:07.270818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:07.271798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:07.272795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:07.272795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:07.272795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:01:07.272795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:01:07.272795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:01:07.272795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:01:07.272795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:01:07.272795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:01:07.272795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 21:01:07.272795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:07.272795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 421/839] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:01:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:07.279938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20698370-8d9b-48fa-9fc5-85bc26671e01 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:07.393676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20698370-8d9b-48fa-9fc5-85bc26671e01 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:07.444754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e3452-cd91-42bf-a4c5-e6b44af52c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:07.445800 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e3452-cd91-42bf-a4c5-e6b44af52c92 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:07.446293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e3452-cd91-42bf-a4c5-e6b44af52c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:07.446778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e3452-cd91-42bf-a4c5-e6b44af52c92 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:07.448111 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e3452-cd91-42bf-a4c5-e6b44af52c92 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:07.448577 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e3452-cd91-42bf-a4c5-e6b44af52c92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:07.451134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e3452-cd91-42bf-a4c5-e6b44af52c92 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:07.474206 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 419/840] 10.222.0.79 () {56 vars in 1135 bytes} [Tue Sep 6 21:01:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:07.479877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26209e4b-62ea-4f93-88c0-4ad8136f3334 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:07.539965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce2011-5b5f-4864-aa2c-2421a2afd158 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:07.540345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce2011-5b5f-4864-aa2c-2421a2afd158 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:07.540510 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce2011-5b5f-4864-aa2c-2421a2afd158 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:07.542192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 420/841] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:07.547563 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e08cb-dd3e-4e83-9f75-9de9540c3663 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:07.547934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e08cb-dd3e-4e83-9f75-9de9540c3663 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:07.548137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e08cb-dd3e-4e83-9f75-9de9540c3663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:07.569719 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-299e08cb-dd3e-4e83-9f75-9de9540c3663 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:07.577770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-299e08cb-dd3e-4e83-9f75-9de9540c3663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:07.586102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:07.592883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:07.619379 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26209e4b-62ea-4f93-88c0-4ad8136f3334 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:07.663985 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321714c9-0d1b-4968-bb2d-758f0529d9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:07.665131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321714c9-0d1b-4968-bb2d-758f0529d9be None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:07.665610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321714c9-0d1b-4968-bb2d-758f0529d9be None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:07.666078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321714c9-0d1b-4968-bb2d-758f0529d9be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:07.677591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 421/842] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:07.747528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321714c9-0d1b-4968-bb2d-758f0529d9be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:07.749373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321714c9-0d1b-4968-bb2d-758f0529d9be None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:07.835570 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:07.861740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 422/843] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:01:07 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:01:07.869835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e08cb-dd3e-4e83-9f75-9de9540c3663 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:07.902439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8d57b-ecea-41d8-83a1-3b0f6bdfda72 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:07.902810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8d57b-ecea-41d8-83a1-3b0f6bdfda72 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:07.903022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8d57b-ecea-41d8-83a1-3b0f6bdfda72 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:07.904675 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 423/844] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:01:07.909494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fea65-60cd-4343-b0e3-cf3f415a1191 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:07.909769 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fea65-60cd-4343-b0e3-cf3f415a1191 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:07.910036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fea65-60cd-4343-b0e3-cf3f415a1191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:07.928849 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-845fea65-60cd-4343-b0e3-cf3f415a1191 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:07.929942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-845fea65-60cd-4343-b0e3-cf3f415a1191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:07.990940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e08cb-dd3e-4e83-9f75-9de9540c3663 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:08.020681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 424/845] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:08.020681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845fea65-60cd-4343-b0e3-cf3f415a1191 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:08.038782 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be58028-a224-44cf-abb2-80adfbd11b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:08.039418 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be58028-a224-44cf-abb2-80adfbd11b29 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:08.039695 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be58028-a224-44cf-abb2-80adfbd11b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:08.039962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be58028-a224-44cf-abb2-80adfbd11b29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:08.049399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:08.052142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be58028-a224-44cf-abb2-80adfbd11b29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:08.053730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be58028-a224-44cf-abb2-80adfbd11b29 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0be58028-a224-44cf-abb2-80adfbd11b29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:08.070574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:08.071578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:08.072456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:08.072456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:08.072456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:08.072456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 422/846] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:08.077684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be58028-a224-44cf-abb2-80adfbd11b29 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:08.143176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845fea65-60cd-4343-b0e3-cf3f415a1191 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:08.145615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:08.181063 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02910d41-6567-46e1-8d0e-9f828c2f4c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:08.181563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02910d41-6567-46e1-8d0e-9f828c2f4c99 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:08.181898 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02910d41-6567-46e1-8d0e-9f828c2f4c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:08.182172 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02910d41-6567-46e1-8d0e-9f828c2f4c99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:08.191084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be58028-a224-44cf-abb2-80adfbd11b29 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:08.193415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02910d41-6567-46e1-8d0e-9f828c2f4c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-02910d41-6567-46e1-8d0e-9f828c2f4c99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:08.198673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:08.199708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:08.201686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:08.201686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:08.201686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:08.201686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:08.201686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:08.201686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:08.201686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:08.201686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:08.201686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:08.201686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:08.201686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:01:08.201686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:08.201686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 425/847] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:01:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:08.205639 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02910d41-6567-46e1-8d0e-9f828c2f4c99 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:08.232708 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a081908-bdf5-45d6-80cc-04df37fa4b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:08.233323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a081908-bdf5-45d6-80cc-04df37fa4b17 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:08.233597 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a081908-bdf5-45d6-80cc-04df37fa4b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:08.233861 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a081908-bdf5-45d6-80cc-04df37fa4b17 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:08.234512 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a081908-bdf5-45d6-80cc-04df37fa4b17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:08.234853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a081908-bdf5-45d6-80cc-04df37fa4b17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:08.236163 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a081908-bdf5-45d6-80cc-04df37fa4b17 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:08.249007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 423/848] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:08.254407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a081908-bdf5-45d6-80cc-04df37fa4b17 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:08.312147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:08.321097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02910d41-6567-46e1-8d0e-9f828c2f4c99 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:08.360940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8602365-1aaa-4b1c-ad07-ba94e4d862dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:08.361552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8602365-1aaa-4b1c-ad07-ba94e4d862dc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:08.361844 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8602365-1aaa-4b1c-ad07-ba94e4d862dc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:08.362089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8602365-1aaa-4b1c-ad07-ba94e4d862dc None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:08.362750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8602365-1aaa-4b1c-ad07-ba94e4d862dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:08.363048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8602365-1aaa-4b1c-ad07-ba94e4d862dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:08.364427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8602365-1aaa-4b1c-ad07-ba94e4d862dc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:08.365375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a081908-bdf5-45d6-80cc-04df37fa4b17 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:08.377604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 426/849] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:01:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:08.377604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8602365-1aaa-4b1c-ad07-ba94e4d862dc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:08.406661 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:08.408778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8e52d-5d2a-4481-9e28-07c5c62b753b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:08.409680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8e52d-5d2a-4481-9e28-07c5c62b753b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:08.409972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8e52d-5d2a-4481-9e28-07c5c62b753b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:08.410319 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8e52d-5d2a-4481-9e28-07c5c62b753b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:08.417159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8e52d-5d2a-4481-9e28-07c5c62b753b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:08.419025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8e52d-5d2a-4481-9e28-07c5c62b753b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8ef8e52d-5d2a-4481-9e28-07c5c62b753b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:01:08.431836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:08.432972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:08.434123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:01:08.434123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:01:08.434123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:01:08.434123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:01:08.434123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:01:08.434123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:01:08.434123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:08.434123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:08.435880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 424/850] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:08.436584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8e52d-5d2a-4481-9e28-07c5c62b753b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:08.492509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8602365-1aaa-4b1c-ad07-ba94e4d862dc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:08.531642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97290349-54e7-4872-a9f4-b9afd9b42f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:08.532280 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97290349-54e7-4872-a9f4-b9afd9b42f54 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:08.532551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97290349-54e7-4872-a9f4-b9afd9b42f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:08.532845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97290349-54e7-4872-a9f4-b9afd9b42f54 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:08.533422 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97290349-54e7-4872-a9f4-b9afd9b42f54 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:08.534860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97290349-54e7-4872-a9f4-b9afd9b42f54 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:08.543161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8e52d-5d2a-4481-9e28-07c5c62b753b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-97290349-54e7-4872-a9f4-b9afd9b42f54 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:08.544043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:08.545084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:08.545933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 427/851] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:01:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:08.546759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97290349-54e7-4872-a9f4-b9afd9b42f54 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:08.561007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:08.590284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eb537-ed04-44b1-95e5-a70b82abec27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:08.590930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eb537-ed04-44b1-95e5-a70b82abec27 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:08.591214 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eb537-ed04-44b1-95e5-a70b82abec27 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:08.591997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eb537-ed04-44b1-95e5-a70b82abec27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:08.603727 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833eb537-ed04-44b1-95e5-a70b82abec27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:08.605233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833eb537-ed04-44b1-95e5-a70b82abec27 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-833eb537-ed04-44b1-95e5-a70b82abec27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:08.623248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:08.624323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:08.625421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:08.625421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:08.625421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:08.625598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 425/852] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:08.626479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eb537-ed04-44b1-95e5-a70b82abec27 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:08.663144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:08.680229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97290349-54e7-4872-a9f4-b9afd9b42f54 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:08.715999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8614d9f-262d-480e-8eb0-88ab9e8b6cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:08.716528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8614d9f-262d-480e-8eb0-88ab9e8b6cff None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:08.716835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8614d9f-262d-480e-8eb0-88ab9e8b6cff None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:08.717104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8614d9f-262d-480e-8eb0-88ab9e8b6cff None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:08.717792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8614d9f-262d-480e-8eb0-88ab9e8b6cff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:08.718171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8614d9f-262d-480e-8eb0-88ab9e8b6cff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:08.719729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8614d9f-262d-480e-8eb0-88ab9e8b6cff None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:08.731815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 428/853] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:08.732620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8614d9f-262d-480e-8eb0-88ab9e8b6cff None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:08.750495 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eb537-ed04-44b1-95e5-a70b82abec27 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:08.785650 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908b02d-76b3-4fa6-b91d-6bda3858c719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:08.786326 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908b02d-76b3-4fa6-b91d-6bda3858c719 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:08.786664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908b02d-76b3-4fa6-b91d-6bda3858c719 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:08.786911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908b02d-76b3-4fa6-b91d-6bda3858c719 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:08.787567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908b02d-76b3-4fa6-b91d-6bda3858c719 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:08.787901 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908b02d-76b3-4fa6-b91d-6bda3858c719 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:08.789449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908b02d-76b3-4fa6-b91d-6bda3858c719 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:08.794278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 426/854] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:01:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:08.795253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908b02d-76b3-4fa6-b91d-6bda3858c719 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:08.825382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:08.836269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8614d9f-262d-480e-8eb0-88ab9e8b6cff None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:08.870816 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b1086-3a35-4bc2-8d31-f0c48066c2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:08.871457 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b1086-3a35-4bc2-8d31-f0c48066c2fb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:08.871730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b1086-3a35-4bc2-8d31-f0c48066c2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:08.872002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b1086-3a35-4bc2-8d31-f0c48066c2fb None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:08.872703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b1086-3a35-4bc2-8d31-f0c48066c2fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:08.872998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b1086-3a35-4bc2-8d31-f0c48066c2fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:08.874377 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b1086-3a35-4bc2-8d31-f0c48066c2fb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:08.885887 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 429/855] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:08.887355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140b45e-4105-4df1-8343-f3756158f514 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:08.887857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140b45e-4105-4df1-8343-f3756158f514 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:08.888301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140b45e-4105-4df1-8343-f3756158f514 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:08.889755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 430/856] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:08.890444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140b45e-4105-4df1-8343-f3756158f514 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:08.902535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908b02d-76b3-4fa6-b91d-6bda3858c719 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:08.902889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:08.938752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82ef2e-a551-4915-8eb0-e6523b37b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:08.939354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82ef2e-a551-4915-8eb0-e6523b37b6cc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:08.939633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82ef2e-a551-4915-8eb0-e6523b37b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:08.939900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82ef2e-a551-4915-8eb0-e6523b37b6cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:08.940622 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82ef2e-a551-4915-8eb0-e6523b37b6cc None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=061c584de8264868af8782402a98a72e) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:08.940962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82ef2e-a551-4915-8eb0-e6523b37b6cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:08.942396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82ef2e-a551-4915-8eb0-e6523b37b6cc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:08.948446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 427/857] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:01:08 2022] GET /identity/v3/role_assignments?user.id=061c584de8264868af8782402a98a72e&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=3b30504991194002b41f8587d0851c10 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:08.949328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82ef2e-a551-4915-8eb0-e6523b37b6cc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:09.019812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140b45e-4105-4df1-8343-f3756158f514 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:09.062010 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fa43a-fa47-4265-98e9-9079326c2132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:09.063148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fa43a-fa47-4265-98e9-9079326c2132 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:09.063799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:09.065266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fa43a-fa47-4265-98e9-9079326c2132 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:09.066072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fa43a-fa47-4265-98e9-9079326c2132 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:09.066739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fa43a-fa47-4265-98e9-9079326c2132 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:09.067056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fa43a-fa47-4265-98e9-9079326c2132 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:09.069067 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fa43a-fa47-4265-98e9-9079326c2132 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:09.074184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82ef2e-a551-4915-8eb0-e6523b37b6cc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:09.084959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 431/858] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:01:08 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:09.084959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07563f0e-6afa-4d7c-85ab-3ed9d000084b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:09.084959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07563f0e-6afa-4d7c-85ab-3ed9d000084b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:09.084959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07563f0e-6afa-4d7c-85ab-3ed9d000084b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:09.108250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-07563f0e-6afa-4d7c-85ab-3ed9d000084b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:09.110607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-07563f0e-6afa-4d7c-85ab-3ed9d000084b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:09.123781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34525f9-bce5-49fa-b561-5c79c758ac37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:09.124601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34525f9-bce5-49fa-b561-5c79c758ac37 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:09.125018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34525f9-bce5-49fa-b561-5c79c758ac37 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:09.125379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34525f9-bce5-49fa-b561-5c79c758ac37 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:09.135473 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34525f9-bce5-49fa-b561-5c79c758ac37 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:09.137080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34525f9-bce5-49fa-b561-5c79c758ac37 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d34525f9-bce5-49fa-b561-5c79c758ac37 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:01:09.151119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:09.152145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:09.153397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:09.153397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:09.153397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:01:09.153397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:01:09.153397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:01:09.153397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:01:09.153397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:01:09.153397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:01:09.153397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 21:01:09.153397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:09.153397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 428/859] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Sep 6 21:01:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:09.159089 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34525f9-bce5-49fa-b561-5c79c758ac37 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:09.203596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 432/860] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:09.292342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34525f9-bce5-49fa-b561-5c79c758ac37 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:09.328195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6fc3f-a710-43eb-92df-d8df289726fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:09.328828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6fc3f-a710-43eb-92df-d8df289726fa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:09.329104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6fc3f-a710-43eb-92df-d8df289726fa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:09.329373 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6fc3f-a710-43eb-92df-d8df289726fa None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:09.330104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6fc3f-a710-43eb-92df-d8df289726fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:09.330402 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6fc3f-a710-43eb-92df-d8df289726fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:09.331937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6fc3f-a710-43eb-92df-d8df289726fa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:09.350307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 429/861] 10.222.0.79 () {56 vars in 1149 bytes} [Tue Sep 6 21:01:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:09.357886 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07563f0e-6afa-4d7c-85ab-3ed9d000084b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:09.361684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:09.466718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07563f0e-6afa-4d7c-85ab-3ed9d000084b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:09.475884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6fc3f-a710-43eb-92df-d8df289726fa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:09.491906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:09.509469 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0e034-a66f-40f7-9528-6ff3d4162156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:09.510203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0e034-a66f-40f7-9528-6ff3d4162156 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:09.510665 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0e034-a66f-40f7-9528-6ff3d4162156 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:09.511017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0e034-a66f-40f7-9528-6ff3d4162156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:09.612273 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0e034-a66f-40f7-9528-6ff3d4162156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:09.614241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0e034-a66f-40f7-9528-6ff3d4162156 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:09.616922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6fc3f-a710-43eb-92df-d8df289726fa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:09.618182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:09.663658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567394e0-ed9f-4dad-86a8-1de79757252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:09.665122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567394e0-ed9f-4dad-86a8-1de79757252e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:09.665546 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567394e0-ed9f-4dad-86a8-1de79757252e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:09.666352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567394e0-ed9f-4dad-86a8-1de79757252e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:09.667392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567394e0-ed9f-4dad-86a8-1de79757252e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:09.669249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567394e0-ed9f-4dad-86a8-1de79757252e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-567394e0-ed9f-4dad-86a8-1de79757252e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:09.682752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:09.683752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 21:01:09.684936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:09.691353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 430/862] 10.222.0.79 () {56 vars in 1132 bytes} [Tue Sep 6 21:01:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:09.692188 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567394e0-ed9f-4dad-86a8-1de79757252e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:09.751464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 433/863] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:01:09 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:01:09.756564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0e034-a66f-40f7-9528-6ff3d4162156 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:09.789812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:09.817791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567394e0-ed9f-4dad-86a8-1de79757252e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:09.854297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1473cd-c01f-436e-937a-85ec83017481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:09.854738 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1473cd-c01f-436e-937a-85ec83017481 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:09.855020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1473cd-c01f-436e-937a-85ec83017481 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:09.855288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1473cd-c01f-436e-937a-85ec83017481 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:09.855980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1473cd-c01f-436e-937a-85ec83017481 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:09.856288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1473cd-c01f-436e-937a-85ec83017481 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:09.857719 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1473cd-c01f-436e-937a-85ec83017481 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:09.868143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 431/864] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Sep 6 21:01:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:09.869682 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1473cd-c01f-436e-937a-85ec83017481 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:09.887148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0e034-a66f-40f7-9528-6ff3d4162156 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:09.890331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:09.937080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17ea82-8d84-4e23-b196-51ee75274ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:09.938123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17ea82-8d84-4e23-b196-51ee75274ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:09.938654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17ea82-8d84-4e23-b196-51ee75274ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:09.939126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17ea82-8d84-4e23-b196-51ee75274ee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:09.954664 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17ea82-8d84-4e23-b196-51ee75274ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:09.956632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17ea82-8d84-4e23-b196-51ee75274ee8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ba17ea82-8d84-4e23-b196-51ee75274ee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:09.974441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:09.975453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:09.976376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:09.976376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:09.976376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:09.978633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 434/865] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:09.982152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1473cd-c01f-436e-937a-85ec83017481 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:09.984806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17ea82-8d84-4e23-b196-51ee75274ee8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:10.027030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4bc6e-08b8-420d-9f15-0db981025e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:10.028267 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4bc6e-08b8-420d-9f15-0db981025e93 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:10.029008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4bc6e-08b8-420d-9f15-0db981025e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:10.029532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4bc6e-08b8-420d-9f15-0db981025e93 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:10.030922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4bc6e-08b8-420d-9f15-0db981025e93 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:10.031508 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4bc6e-08b8-420d-9f15-0db981025e93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:10.033137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4bc6e-08b8-420d-9f15-0db981025e93 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:10.046899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 432/866] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Sep 6 21:01:09 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:10.126099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17ea82-8d84-4e23-b196-51ee75274ee8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:10.165381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fccd043-ef11-4df3-9b6e-f688b3d028d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:10.166332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccd043-ef11-4df3-9b6e-f688b3d028d3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:10.166784 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccd043-ef11-4df3-9b6e-f688b3d028d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:10.167249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccd043-ef11-4df3-9b6e-f688b3d028d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:10.168086 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fccd043-ef11-4df3-9b6e-f688b3d028d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:10.168650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fccd043-ef11-4df3-9b6e-f688b3d028d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:10.170379 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fccd043-ef11-4df3-9b6e-f688b3d028d3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:10.185461 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 435/867] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:10.193946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4bc6e-08b8-420d-9f15-0db981025e93 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:10.197619 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:10.312562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4bc6e-08b8-420d-9f15-0db981025e93 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:10.349848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e38cd-f6da-4a75-9bdd-51921f3430b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:10.350925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e38cd-f6da-4a75-9bdd-51921f3430b3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:10.351305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e38cd-f6da-4a75-9bdd-51921f3430b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:10.351729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e38cd-f6da-4a75-9bdd-51921f3430b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:10.361258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e38cd-f6da-4a75-9bdd-51921f3430b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:10.363064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e38cd-f6da-4a75-9bdd-51921f3430b3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-539e38cd-f6da-4a75-9bdd-51921f3430b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:01:10.376789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:10.377879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:10.378915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:01:10.378915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:01:10.378915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:01:10.378915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:01:10.378915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:01:10.378915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:01:10.378915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:10.378915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:10.385049 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 433/868] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:10.388120 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fccd043-ef11-4df3-9b6e-f688b3d028d3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:10.410796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:10.510420 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fccd043-ef11-4df3-9b6e-f688b3d028d3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:10.556284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e185028-bec0-477a-9f11-9536c2304260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:10.557188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e185028-bec0-477a-9f11-9536c2304260 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:10.557565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e185028-bec0-477a-9f11-9536c2304260 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:10.557935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e185028-bec0-477a-9f11-9536c2304260 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:10.558736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e185028-bec0-477a-9f11-9536c2304260 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:10.559453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e185028-bec0-477a-9f11-9536c2304260 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:10.561467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e185028-bec0-477a-9f11-9536c2304260 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:10.577171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 436/869] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:01:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:10.583239 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e38cd-f6da-4a75-9bdd-51921f3430b3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:10.651250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:10.696475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e38cd-f6da-4a75-9bdd-51921f3430b3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:10.734789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974ec93-44e0-4097-ac93-ac6ee0fe55d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:10.735653 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974ec93-44e0-4097-ac93-ac6ee0fe55d8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:10.736072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974ec93-44e0-4097-ac93-ac6ee0fe55d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:10.736506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974ec93-44e0-4097-ac93-ac6ee0fe55d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:10.737388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974ec93-44e0-4097-ac93-ac6ee0fe55d8 None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=c09b4900d97b4b35b617af9a42baf9ee) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:10.737848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974ec93-44e0-4097-ac93-ac6ee0fe55d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:10.739499 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974ec93-44e0-4097-ac93-ac6ee0fe55d8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:10.746734 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 434/870] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:01:10 2022] GET /identity/v3/role_assignments?user.id=c09b4900d97b4b35b617af9a42baf9ee&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:11.330165 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab85be-e82f-4a3f-8692-2b3c69c4b590 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:11.330696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab85be-e82f-4a3f-8692-2b3c69c4b590 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:11.330696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab85be-e82f-4a3f-8692-2b3c69c4b590 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:11.332320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 437/871] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:11.338432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcea0ca-e440-412b-a3c2-2083daa6aeb3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:11.338890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcea0ca-e440-412b-a3c2-2083daa6aeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:11.338979 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcea0ca-e440-412b-a3c2-2083daa6aeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:11.358079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7dcea0ca-e440-412b-a3c2-2083daa6aeb3 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:11.359269 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7dcea0ca-e440-412b-a3c2-2083daa6aeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:11.417680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31259bb6-bd3b-4d95-9f8a-cc7f5597c8a4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:11.417989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31259bb6-bd3b-4d95-9f8a-cc7f5597c8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:11.418277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31259bb6-bd3b-4d95-9f8a-cc7f5597c8a4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:11.419696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 438/872] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:11.425025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592de8a-29e8-4c76-a943-a5382c472033 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:11.425306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592de8a-29e8-4c76-a943-a5382c472033 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:11.425578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592de8a-29e8-4c76-a943-a5382c472033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:11.437150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 435/873] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:11.445856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6592de8a-29e8-4c76-a943-a5382c472033 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:11.447069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6592de8a-29e8-4c76-a943-a5382c472033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:11.522438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 439/874] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:11.722106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcea0ca-e440-412b-a3c2-2083daa6aeb3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:11.784090 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:11.806406 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592de8a-29e8-4c76-a943-a5382c472033 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:11.815470 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:11.841265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcea0ca-e440-412b-a3c2-2083daa6aeb3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:11.882701 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f6de3-91ba-40e7-ab09-220888c85754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:11.883343 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f6de3-91ba-40e7-ab09-220888c85754 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:11.883687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f6de3-91ba-40e7-ab09-220888c85754 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:11.883959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f6de3-91ba-40e7-ab09-220888c85754 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:11.884765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f6de3-91ba-40e7-ab09-220888c85754 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:11.886830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f6de3-91ba-40e7-ab09-220888c85754 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-599f6de3-91ba-40e7-ab09-220888c85754 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:11.897684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:11.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:11.899538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:11.904757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 436/875] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Sep 6 21:01:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:11.905631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f6de3-91ba-40e7-ab09-220888c85754 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:11.935290 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592de8a-29e8-4c76-a943-a5382c472033 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:11.979833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcee36-274f-48c4-8f1e-9950bd54f88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:11.981319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcee36-274f-48c4-8f1e-9950bd54f88a None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:11.981523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcee36-274f-48c4-8f1e-9950bd54f88a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:11.982072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcee36-274f-48c4-8f1e-9950bd54f88a None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:11.982730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcee36-274f-48c4-8f1e-9950bd54f88a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:11.984090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcee36-274f-48c4-8f1e-9950bd54f88a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:12.007427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 440/876] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 21:01:11 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:12.036425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f6de3-91ba-40e7-ab09-220888c85754 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:12.060878 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:12.084900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c9da6-b44f-41c2-b254-d74c9d7e3d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:12.085954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c9da6-b44f-41c2-b254-d74c9d7e3d5e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:12.086454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c9da6-b44f-41c2-b254-d74c9d7e3d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:12.086947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c9da6-b44f-41c2-b254-d74c9d7e3d5e None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:12.087921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c9da6-b44f-41c2-b254-d74c9d7e3d5e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:12.088450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c9da6-b44f-41c2-b254-d74c9d7e3d5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:12.090170 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c9da6-b44f-41c2-b254-d74c9d7e3d5e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:12.096463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 437/877] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Sep 6 21:01:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:12.104870 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcee36-274f-48c4-8f1e-9950bd54f88a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:12.176109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:12.215365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcee36-274f-48c4-8f1e-9950bd54f88a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:12.253628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f1e66-5539-46d8-8f77-e2303b5571e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:12.254263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f1e66-5539-46d8-8f77-e2303b5571e5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:12.254539 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f1e66-5539-46d8-8f77-e2303b5571e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:12.254808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f1e66-5539-46d8-8f77-e2303b5571e5 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:12.255456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f1e66-5539-46d8-8f77-e2303b5571e5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:12.255758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f1e66-5539-46d8-8f77-e2303b5571e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:12.257225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f1e66-5539-46d8-8f77-e2303b5571e5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:12.263863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 441/878] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Sep 6 21:01:12 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:12.365265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92405002-bf8e-409f-b655-01e4921be5e5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:12.365821 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92405002-bf8e-409f-b655-01e4921be5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:12.365821 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92405002-bf8e-409f-b655-01e4921be5e5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:12.367434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 438/879] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:12.373595 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887e3dc-739f-4e7e-910e-e2af66b2019b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:12.374105 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887e3dc-739f-4e7e-910e-e2af66b2019b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:12.374213 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887e3dc-739f-4e7e-910e-e2af66b2019b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:12.394934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c887e3dc-739f-4e7e-910e-e2af66b2019b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:12.396187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c887e3dc-739f-4e7e-910e-e2af66b2019b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:12.475728 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 442/880] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:12.747493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92405002-bf8e-409f-b655-01e4921be5e5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:12.856934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92405002-bf8e-409f-b655-01e4921be5e5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:12.892871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ccf60-718a-4f1d-9305-e44aa520bc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:12.893921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ccf60-718a-4f1d-9305-e44aa520bc7b None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:12.894417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ccf60-718a-4f1d-9305-e44aa520bc7b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:12.894904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ccf60-718a-4f1d-9305-e44aa520bc7b None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:12.895809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ccf60-718a-4f1d-9305-e44aa520bc7b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:12.897483 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ccf60-718a-4f1d-9305-e44aa520bc7b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:12.917177 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 439/881] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 21:01:12 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:13.029611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8aed9-ef99-494b-a18e-23c5dc24ed82 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:13.029832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8aed9-ef99-494b-a18e-23c5dc24ed82 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:13.030067 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8aed9-ef99-494b-a18e-23c5dc24ed82 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:13.031725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 443/882] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:13.037485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d385f54-397a-4400-8f7b-5cce24aca04e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:13.038090 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d385f54-397a-4400-8f7b-5cce24aca04e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:13.038567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d385f54-397a-4400-8f7b-5cce24aca04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:13.059361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0d385f54-397a-4400-8f7b-5cce24aca04e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:13.061509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0d385f54-397a-4400-8f7b-5cce24aca04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:13.101364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:13.144246 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 440/883] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:13.423276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8aed9-ef99-494b-a18e-23c5dc24ed82 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:13.489188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:13.532204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8aed9-ef99-494b-a18e-23c5dc24ed82 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:13.567104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a903d7c-bb90-4316-9d0e-ebb208a3f0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:13.568000 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a903d7c-bb90-4316-9d0e-ebb208a3f0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:13.568414 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a903d7c-bb90-4316-9d0e-ebb208a3f0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:13.568861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a903d7c-bb90-4316-9d0e-ebb208a3f0f8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:13.579835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a903d7c-bb90-4316-9d0e-ebb208a3f0f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0a903d7c-bb90-4316-9d0e-ebb208a3f0f8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:13.585274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:13.586567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:13.588072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:13.588072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:13.588072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:13.588072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:13.588072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:13.588072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:13.588072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:13.588072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:13.588072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:13.588072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:13.588072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 21:01:13.588072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:13.590466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 444/884] 10.222.0.79 () {56 vars in 1120 bytes} [Tue Sep 6 21:01:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:13.596803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d385f54-397a-4400-8f7b-5cce24aca04e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:13.698452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d385f54-397a-4400-8f7b-5cce24aca04e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:13.733936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bcf19-a0e2-4806-a701-63956ab914fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:13.734543 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bcf19-a0e2-4806-a701-63956ab914fa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:13.734824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bcf19-a0e2-4806-a701-63956ab914fa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:13.735109 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bcf19-a0e2-4806-a701-63956ab914fa None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:13.735790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bcf19-a0e2-4806-a701-63956ab914fa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:13.736088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bcf19-a0e2-4806-a701-63956ab914fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:13.737562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bcf19-a0e2-4806-a701-63956ab914fa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:13.751976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 441/885] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Sep 6 21:01:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:13.753791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a903d7c-bb90-4316-9d0e-ebb208a3f0f8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:13.862703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a903d7c-bb90-4316-9d0e-ebb208a3f0f8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:13.885585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:13.899579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ac5f1-8b46-4799-8a7f-2ef256babbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:13.900342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ac5f1-8b46-4799-8a7f-2ef256babbbe None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:13.900613 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ac5f1-8b46-4799-8a7f-2ef256babbbe None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:13.900908 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ac5f1-8b46-4799-8a7f-2ef256babbbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:13.911145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ac5f1-8b46-4799-8a7f-2ef256babbbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:13.912902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ac5f1-8b46-4799-8a7f-2ef256babbbe None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2f1ac5f1-8b46-4799-8a7f-2ef256babbbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:13.927928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:13.929029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:13.930182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:13.930182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:13.930182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:13.930182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 445/886] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:13.938071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bcf19-a0e2-4806-a701-63956ab914fa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:13.970521 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:14.050660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bcf19-a0e2-4806-a701-63956ab914fa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:14.085471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e4beb-0e8a-4acf-9fc2-0452c1476673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:14.086076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e4beb-0e8a-4acf-9fc2-0452c1476673 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:14.086537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e4beb-0e8a-4acf-9fc2-0452c1476673 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:14.086802 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e4beb-0e8a-4acf-9fc2-0452c1476673 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:14.087439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e4beb-0e8a-4acf-9fc2-0452c1476673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:14.087735 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e4beb-0e8a-4acf-9fc2-0452c1476673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:14.089130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e4beb-0e8a-4acf-9fc2-0452c1476673 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:14.101545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 442/887] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:14.109809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ac5f1-8b46-4799-8a7f-2ef256babbbe None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:14.218645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ac5f1-8b46-4799-8a7f-2ef256babbbe None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:14.258071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907037d9-362e-4a45-8d34-ca07f60a0531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:14.258704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907037d9-362e-4a45-8d34-ca07f60a0531 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:14.258975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907037d9-362e-4a45-8d34-ca07f60a0531 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:14.259242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907037d9-362e-4a45-8d34-ca07f60a0531 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:14.267271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907037d9-362e-4a45-8d34-ca07f60a0531 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:14.268731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907037d9-362e-4a45-8d34-ca07f60a0531 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-907037d9-362e-4a45-8d34-ca07f60a0531 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:01:14.280347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:14.281361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:14.282494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:14.282494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:14.282494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:01:14.282494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:01:14.282494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:01:14.282494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:01:14.282494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:01:14.282494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:01:14.282494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 21:01:14.282494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:14.282494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 446/888] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Sep 6 21:01:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:14.289845 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e4beb-0e8a-4acf-9fc2-0452c1476673 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:14.304266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddb9b8-fe2e-46a7-8344-941b2c151ce5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:14.304562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddb9b8-fe2e-46a7-8344-941b2c151ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:14.304870 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddb9b8-fe2e-46a7-8344-941b2c151ce5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:14.307055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 447/889] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:01:14.313142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457e7df-9598-4f53-afe2-5a1bb4d13864 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:14.313427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457e7df-9598-4f53-afe2-5a1bb4d13864 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:14.313718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457e7df-9598-4f53-afe2-5a1bb4d13864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:14.332917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f457e7df-9598-4f53-afe2-5a1bb4d13864 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:14.334116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f457e7df-9598-4f53-afe2-5a1bb4d13864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:14.364328 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:14.411094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:14.426926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e4beb-0e8a-4acf-9fc2-0452c1476673 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:14.428747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 448/890] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:14.467744 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b003cc-5a1c-4cd1-8263-5e24b04364d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:14.468360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b003cc-5a1c-4cd1-8263-5e24b04364d3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:14.468758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b003cc-5a1c-4cd1-8263-5e24b04364d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:14.469026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b003cc-5a1c-4cd1-8263-5e24b04364d3 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:14.469755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b003cc-5a1c-4cd1-8263-5e24b04364d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:14.470060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b003cc-5a1c-4cd1-8263-5e24b04364d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:14.471496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b003cc-5a1c-4cd1-8263-5e24b04364d3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:14.485898 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a94dd-04ea-4ce1-a2e3-97066ec9966b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:14.486570 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a94dd-04ea-4ce1-a2e3-97066ec9966b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:14.486901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a94dd-04ea-4ce1-a2e3-97066ec9966b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:14.488724 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 449/891] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:01:14.492928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 443/892] 10.222.0.79 () {56 vars in 1149 bytes} [Tue Sep 6 21:01:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:14.494485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88d804-8c55-4fe7-bc75-436282b14866 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:14.494993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88d804-8c55-4fe7-bc75-436282b14866 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:14.495473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88d804-8c55-4fe7-bc75-436282b14866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:14.495677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b003cc-5a1c-4cd1-8263-5e24b04364d3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:14.516885 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3d88d804-8c55-4fe7-bc75-436282b14866 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:14.518399 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3d88d804-8c55-4fe7-bc75-436282b14866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:14.609028 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 450/893] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:14.622507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b003cc-5a1c-4cd1-8263-5e24b04364d3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:14.657971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37123d8a-7584-40b0-b840-55bfd2ed5dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:14.658584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37123d8a-7584-40b0-b840-55bfd2ed5dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:14.658857 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37123d8a-7584-40b0-b840-55bfd2ed5dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:14.659130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37123d8a-7584-40b0-b840-55bfd2ed5dd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:14.675169 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:14.729541 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88d804-8c55-4fe7-bc75-436282b14866 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:14.732399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37123d8a-7584-40b0-b840-55bfd2ed5dd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:14.733904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37123d8a-7584-40b0-b840-55bfd2ed5dd1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:14.843576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:14.853818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88d804-8c55-4fe7-bc75-436282b14866 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:14.859183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 444/894] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:01:14 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:01:14.865563 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37123d8a-7584-40b0-b840-55bfd2ed5dd1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:14.905142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d9650-997b-4758-a72b-81b3732925a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:14.905783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d9650-997b-4758-a72b-81b3732925a7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:14.906054 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d9650-997b-4758-a72b-81b3732925a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:14.906318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d9650-997b-4758-a72b-81b3732925a7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:14.906905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d9650-997b-4758-a72b-81b3732925a7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:14.908740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d9650-997b-4758-a72b-81b3732925a7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:14.914903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-301d9650-997b-4758-a72b-81b3732925a7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:14.918333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:14.919318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 21:01:14.920159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:14.920938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 451/895] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:01:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:14.921565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d9650-997b-4758-a72b-81b3732925a7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:14.978871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37123d8a-7584-40b0-b840-55bfd2ed5dd1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:15.017278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b1329-c745-450c-ae4d-51879e3b0202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:15.017947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b1329-c745-450c-ae4d-51879e3b0202 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:15.018219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b1329-c745-450c-ae4d-51879e3b0202 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:15.018481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b1329-c745-450c-ae4d-51879e3b0202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:15.029164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b1329-c745-450c-ae4d-51879e3b0202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:15.030603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b1329-c745-450c-ae4d-51879e3b0202 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:15.035602 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d9650-997b-4758-a72b-81b3732925a7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-492b1329-c745-450c-ae4d-51879e3b0202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:15.057290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:15.058324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:15.059322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:15.059322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:15.059322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:15.059322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 445/896] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:15.061047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b1329-c745-450c-ae4d-51879e3b0202 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:15.080672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7df840-d58f-46e7-a311-fd3487b0a84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:15.081320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7df840-d58f-46e7-a311-fd3487b0a84a None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:15.081598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7df840-d58f-46e7-a311-fd3487b0a84a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:15.081869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7df840-d58f-46e7-a311-fd3487b0a84a None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:15.082843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7df840-d58f-46e7-a311-fd3487b0a84a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:15.084243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7df840-d58f-46e7-a311-fd3487b0a84a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:15.100094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 452/897] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 21:01:14 2022] POST /identity/v3/services => generated 241 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:15.110096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7df840-d58f-46e7-a311-fd3487b0a84a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:15.145075 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:15.217376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:15.223890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b1329-c745-450c-ae4d-51879e3b0202 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:15.276571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc217c88-0d4a-471b-b4d9-06c48dcc2ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:15.277606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc217c88-0d4a-471b-b4d9-06c48dcc2ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:15.278149 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc217c88-0d4a-471b-b4d9-06c48dcc2ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:15.278599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc217c88-0d4a-471b-b4d9-06c48dcc2ec8 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:15.279615 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc217c88-0d4a-471b-b4d9-06c48dcc2ec8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:15.280115 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc217c88-0d4a-471b-b4d9-06c48dcc2ec8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:15.282216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc217c88-0d4a-471b-b4d9-06c48dcc2ec8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:15.289816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 446/898] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:01:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:15.295564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc217c88-0d4a-471b-b4d9-06c48dcc2ec8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:15.316371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7df840-d58f-46e7-a311-fd3487b0a84a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:15.354017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cfce0-973a-4b31-93de-ab4b485b1b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:15.354553 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cfce0-973a-4b31-93de-ab4b485b1b74 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:15.354826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cfce0-973a-4b31-93de-ab4b485b1b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:15.355097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cfce0-973a-4b31-93de-ab4b485b1b74 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:15.355812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cfce0-973a-4b31-93de-ab4b485b1b74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:15.356116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cfce0-973a-4b31-93de-ab4b485b1b74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:15.357579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cfce0-973a-4b31-93de-ab4b485b1b74 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:15.376704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 453/899] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:15.378248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d532-39ac-4015-9504-00a3892ea2b8 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:15.378534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d532-39ac-4015-9504-00a3892ea2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:15.378803 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d532-39ac-4015-9504-00a3892ea2b8 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:15.380703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 454/900] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:15.382568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438d532-39ac-4015-9504-00a3892ea2b8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:15.438457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc217c88-0d4a-471b-b4d9-06c48dcc2ec8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:15.467649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:15.476528 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601c420-c206-4c53-b622-865990781ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:15.477194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601c420-c206-4c53-b622-865990781ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:15.477470 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601c420-c206-4c53-b622-865990781ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:15.477741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601c420-c206-4c53-b622-865990781ec2 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:15.478390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601c420-c206-4c53-b622-865990781ec2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:15.478691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601c420-c206-4c53-b622-865990781ec2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:15.480080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601c420-c206-4c53-b622-865990781ec2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:15.485593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 447/901] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:01:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:15.491742 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e8431-6a12-4567-a14e-d6f9116b4c26 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:15.492028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e8431-6a12-4567-a14e-d6f9116b4c26 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:15.492304 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e8431-6a12-4567-a14e-d6f9116b4c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:15.495306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438d532-39ac-4015-9504-00a3892ea2b8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:15.510305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ea5e8431-6a12-4567-a14e-d6f9116b4c26 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:15.511512 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ea5e8431-6a12-4567-a14e-d6f9116b4c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:15.529439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:15.533464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdada17-6bea-495f-8bf8-51e6753728db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:15.534111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdada17-6bea-495f-8bf8-51e6753728db None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:15.534388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdada17-6bea-495f-8bf8-51e6753728db None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:15.534641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdada17-6bea-495f-8bf8-51e6753728db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:15.541548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdada17-6bea-495f-8bf8-51e6753728db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:15.542996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdada17-6bea-495f-8bf8-51e6753728db None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7bdada17-6bea-495f-8bf8-51e6753728db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:01:15.555810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:15.557036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:15.563557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:01:15.563557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:01:15.563557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:01:15.563557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:01:15.563557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:01:15.563557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:01:15.563557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:15.563557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:15.563557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 455/902] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:15.563557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdada17-6bea-495f-8bf8-51e6753728db None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:15.604769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 448/903] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:15.604769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e8431-6a12-4567-a14e-d6f9116b4c26 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:15.692110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdada17-6bea-495f-8bf8-51e6753728db None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:15.717798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e8431-6a12-4567-a14e-d6f9116b4c26 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:15.735176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ece579-1e18-44e3-ae86-8d6631a5d640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:15.735922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ece579-1e18-44e3-ae86-8d6631a5d640 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:15.736200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ece579-1e18-44e3-ae86-8d6631a5d640 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:15.736471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ece579-1e18-44e3-ae86-8d6631a5d640 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:15.737370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ece579-1e18-44e3-ae86-8d6631a5d640 None admin] RBAC: Adding query filter params (interface=public, service_id=f4a1e0399c1b4c1faccd977ff6e94117, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:15.737682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ece579-1e18-44e3-ae86-8d6631a5d640 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:15.739377 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ece579-1e18-44e3-ae86-8d6631a5d640 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:15.745192 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:15.750526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 456/904] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Sep 6 21:01:15 2022] GET /identity/v3/endpoints?service_id=f4a1e0399c1b4c1faccd977ff6e94117&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:15.760791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ece579-1e18-44e3-ae86-8d6631a5d640 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:15.764907 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0a949-bc00-4775-9421-4dda17d8984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:15.765574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a949-bc00-4775-9421-4dda17d8984f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:15.765850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a949-bc00-4775-9421-4dda17d8984f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:15.766138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a949-bc00-4775-9421-4dda17d8984f None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:15.766847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0a949-bc00-4775-9421-4dda17d8984f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:15.767154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0a949-bc00-4775-9421-4dda17d8984f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:15.768603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0a949-bc00-4775-9421-4dda17d8984f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:15.792687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 449/905] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:01:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:15.794352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0a949-bc00-4775-9421-4dda17d8984f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:15.813362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:15.875547 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ece579-1e18-44e3-ae86-8d6631a5d640 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:15.907479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0a949-bc00-4775-9421-4dda17d8984f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:15.910753 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd445f-ae7b-429b-9fb4-50ab6aca569d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:15.911379 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd445f-ae7b-429b-9fb4-50ab6aca569d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:15.911654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd445f-ae7b-429b-9fb4-50ab6aca569d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:15.911923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd445f-ae7b-429b-9fb4-50ab6aca569d None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:15.912531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd445f-ae7b-429b-9fb4-50ab6aca569d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:15.913941 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd445f-ae7b-429b-9fb4-50ab6aca569d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:15.928700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 457/906] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Sep 6 21:01:15 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:01:15.928700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd445f-ae7b-429b-9fb4-50ab6aca569d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:15.954555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1a945-aa34-402c-80d1-f2ef715c5302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:15.955388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1a945-aa34-402c-80d1-f2ef715c5302 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:15.955668 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1a945-aa34-402c-80d1-f2ef715c5302 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:15.955935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1a945-aa34-402c-80d1-f2ef715c5302 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:15.956699 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1a945-aa34-402c-80d1-f2ef715c5302 None admin] RBAC: Adding query filter params (role.id=6bc0365fc21845598826c2848897e72b, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=c09b4900d97b4b35b617af9a42baf9ee) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:15.957008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1a945-aa34-402c-80d1-f2ef715c5302 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:15.958623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1a945-aa34-402c-80d1-f2ef715c5302 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:15.966554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 450/907] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:01:15 2022] GET /identity/v3/role_assignments?user.id=c09b4900d97b4b35b617af9a42baf9ee&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=6bc0365fc21845598826c2848897e72b => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:16.007993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:16.063366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd445f-ae7b-429b-9fb4-50ab6aca569d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:16.102828 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208074e1-6db7-4373-bb3b-c359694d6ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:16.103497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208074e1-6db7-4373-bb3b-c359694d6ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:16.103772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208074e1-6db7-4373-bb3b-c359694d6ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:16.104038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208074e1-6db7-4373-bb3b-c359694d6ee3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:16.104641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208074e1-6db7-4373-bb3b-c359694d6ee3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:16.106381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208074e1-6db7-4373-bb3b-c359694d6ee3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-208074e1-6db7-4373-bb3b-c359694d6ee3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:16.116489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:16.117548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:16.118861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 458/908] 10.222.0.79 () {56 vars in 1132 bytes} [Tue Sep 6 21:01:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:16.124410 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1a945-aa34-402c-80d1-f2ef715c5302 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:16.235584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:16.259081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1a945-aa34-402c-80d1-f2ef715c5302 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:16.292836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2caf5-446c-405b-80f3-b0df91c0b0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:16.293507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2caf5-446c-405b-80f3-b0df91c0b0de None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:16.293784 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2caf5-446c-405b-80f3-b0df91c0b0de None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:16.294051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2caf5-446c-405b-80f3-b0df91c0b0de None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:16.294690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2caf5-446c-405b-80f3-b0df91c0b0de None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:16.294989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2caf5-446c-405b-80f3-b0df91c0b0de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:16.296369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2caf5-446c-405b-80f3-b0df91c0b0de None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:16.301573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 451/909] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Sep 6 21:01:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:16.307202 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208074e1-6db7-4373-bb3b-c359694d6ee3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:16.416088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208074e1-6db7-4373-bb3b-c359694d6ee3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:16.444172 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:16.453737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5d419-25a4-48ec-acab-00fb221dd332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:16.454527 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5d419-25a4-48ec-acab-00fb221dd332 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:16.454867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5d419-25a4-48ec-acab-00fb221dd332 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:16.455316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5d419-25a4-48ec-acab-00fb221dd332 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:16.455952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5d419-25a4-48ec-acab-00fb221dd332 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:16.456251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5d419-25a4-48ec-acab-00fb221dd332 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:16.457669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5d419-25a4-48ec-acab-00fb221dd332 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:16.462901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 459/910] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Sep 6 21:01:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:16.468891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2caf5-446c-405b-80f3-b0df91c0b0de None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:16.575684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2caf5-446c-405b-80f3-b0df91c0b0de None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:16.614410 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317a5c7-9e9c-4ff1-a36f-a11f4e48b194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:16.615030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317a5c7-9e9c-4ff1-a36f-a11f4e48b194 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:16.615309 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317a5c7-9e9c-4ff1-a36f-a11f4e48b194 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:16.615580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317a5c7-9e9c-4ff1-a36f-a11f4e48b194 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:16.616293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317a5c7-9e9c-4ff1-a36f-a11f4e48b194 None admin] RBAC: Adding query filter params (interface=public, service_id=2fb3c5cc1474442996d6b71bf08ca015, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:16.616608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317a5c7-9e9c-4ff1-a36f-a11f4e48b194 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:16.618014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317a5c7-9e9c-4ff1-a36f-a11f4e48b194 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:16.623903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 452/911] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Sep 6 21:01:16 2022] GET /identity/v3/endpoints?service_id=2fb3c5cc1474442996d6b71bf08ca015&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:16.629397 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5d419-25a4-48ec-acab-00fb221dd332 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:16.734854 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5d419-25a4-48ec-acab-00fb221dd332 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:16.770229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f37316-a622-479d-a665-20125eb3b446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:16.771218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f37316-a622-479d-a665-20125eb3b446 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:16.771697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f37316-a622-479d-a665-20125eb3b446 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:16.772160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f37316-a622-479d-a665-20125eb3b446 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:16.772990 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f37316-a622-479d-a665-20125eb3b446 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:16.774611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f37316-a622-479d-a665-20125eb3b446 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:16.781476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 460/912] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Sep 6 21:01:16 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:01:17.526243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057a64e-2e8c-4b3b-bf8d-085b9c1ec384 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:17.526805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057a64e-2e8c-4b3b-bf8d-085b9c1ec384 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:17.526805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057a64e-2e8c-4b3b-bf8d-085b9c1ec384 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:17.528463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 453/913] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:17.533961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e37fc-29dd-4f2b-bb9d-1a4d8a0c866a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:17.534242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e37fc-29dd-4f2b-bb9d-1a4d8a0c866a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:17.534505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e37fc-29dd-4f2b-bb9d-1a4d8a0c866a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:17.553877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-419e37fc-29dd-4f2b-bb9d-1a4d8a0c866a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:17.555056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-419e37fc-29dd-4f2b-bb9d-1a4d8a0c866a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:17.617956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:17.639421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 461/914] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:17.910050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057a64e-2e8c-4b3b-bf8d-085b9c1ec384 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:17.914787 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:18.026127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057a64e-2e8c-4b3b-bf8d-085b9c1ec384 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:18.061958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76595b18-1a17-4796-a011-b50233ef4c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:18.062598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76595b18-1a17-4796-a011-b50233ef4c83 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:18.062874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76595b18-1a17-4796-a011-b50233ef4c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:18.063145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76595b18-1a17-4796-a011-b50233ef4c83 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:18.063733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76595b18-1a17-4796-a011-b50233ef4c83 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:18.065144 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76595b18-1a17-4796-a011-b50233ef4c83 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-76595b18-1a17-4796-a011-b50233ef4c83 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:18.074223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:18.075224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:18.076057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 454/915] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Sep 6 21:01:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:18.083598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e37fc-29dd-4f2b-bb9d-1a4d8a0c866a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:18.197029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e37fc-29dd-4f2b-bb9d-1a4d8a0c866a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:18.236135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27803cd2-599c-4f26-b0cd-9b37c2494491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:18.236801 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27803cd2-599c-4f26-b0cd-9b37c2494491 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:18.237058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27803cd2-599c-4f26-b0cd-9b37c2494491 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:18.237322 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27803cd2-599c-4f26-b0cd-9b37c2494491 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:18.237965 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27803cd2-599c-4f26-b0cd-9b37c2494491 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:18.238322 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27803cd2-599c-4f26-b0cd-9b37c2494491 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:18.239725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27803cd2-599c-4f26-b0cd-9b37c2494491 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:18.245325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 462/916] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Sep 6 21:01:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:18.252081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76595b18-1a17-4796-a011-b50233ef4c83 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:18.258141 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf6514-5b5e-45dd-87c4-0936e5569b0a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:18.258809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf6514-5b5e-45dd-87c4-0936e5569b0a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:18.259376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf6514-5b5e-45dd-87c4-0936e5569b0a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:18.261201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 463/917] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:18.266562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9c021-c91a-4e60-8887-90bd30134fdf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:18.266847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9c021-c91a-4e60-8887-90bd30134fdf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:18.267125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9c021-c91a-4e60-8887-90bd30134fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:18.285142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e6a9c021-c91a-4e60-8887-90bd30134fdf None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:18.286286 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e6a9c021-c91a-4e60-8887-90bd30134fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:18.332475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:18.334401 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:18.357883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 464/918] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:18.359242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0e3f0-6d25-4e27-aad0-955ab1bdbf24 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:18.359528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0e3f0-6d25-4e27-aad0-955ab1bdbf24 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:18.359802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0e3f0-6d25-4e27-aad0-955ab1bdbf24 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:18.361357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 465/919] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:18.372168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255f5a7-620d-412a-ae86-f1d87452eb5f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:18.372348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255f5a7-620d-412a-ae86-f1d87452eb5f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:18.372628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255f5a7-620d-412a-ae86-f1d87452eb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:18.375547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76595b18-1a17-4796-a011-b50233ef4c83 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:18.397744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d255f5a7-620d-412a-ae86-f1d87452eb5f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:18.401298 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d255f5a7-620d-412a-ae86-f1d87452eb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:18.411829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdf845-b762-4597-8a6d-5487b7b31d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:18.412607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdf845-b762-4597-8a6d-5487b7b31d22 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:18.413004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdf845-b762-4597-8a6d-5487b7b31d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:18.413373 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdf845-b762-4597-8a6d-5487b7b31d22 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:18.414104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fdf845-b762-4597-8a6d-5487b7b31d22 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:18.414500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fdf845-b762-4597-8a6d-5487b7b31d22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:18.416007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fdf845-b762-4597-8a6d-5487b7b31d22 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:18.426020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 455/920] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Sep 6 21:01:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:18.427523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdf845-b762-4597-8a6d-5487b7b31d22 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:18.484857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 466/921] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:18.539349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdf845-b762-4597-8a6d-5487b7b31d22 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:18.571186 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:18.577781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139b6ad-c173-483f-ac99-bfb47aa4015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:18.578732 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139b6ad-c173-483f-ac99-bfb47aa4015f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:18.579204 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139b6ad-c173-483f-ac99-bfb47aa4015f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:18.579643 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139b6ad-c173-483f-ac99-bfb47aa4015f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:18.580505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139b6ad-c173-483f-ac99-bfb47aa4015f None admin] RBAC: Adding query filter params (interface=public, service_id=56563e6b9bae4ead93a8958fa2f9af5c, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:18.581009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139b6ad-c173-483f-ac99-bfb47aa4015f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:18.582793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139b6ad-c173-483f-ac99-bfb47aa4015f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:18.589036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 456/922] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Sep 6 21:01:18 2022] GET /identity/v3/endpoints?service_id=56563e6b9bae4ead93a8958fa2f9af5c&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:18.594995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255f5a7-620d-412a-ae86-f1d87452eb5f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:18.653314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139b6ad-c173-483f-ac99-bfb47aa4015f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:18.712848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255f5a7-620d-412a-ae86-f1d87452eb5f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:18.728244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:18.759787 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bede435-47d0-46c1-89ff-468f16d3b81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:18.760686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bede435-47d0-46c1-89ff-468f16d3b81f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:18.760966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bede435-47d0-46c1-89ff-468f16d3b81f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:18.761229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bede435-47d0-46c1-89ff-468f16d3b81f None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:18.762017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bede435-47d0-46c1-89ff-468f16d3b81f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:18.763933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bede435-47d0-46c1-89ff-468f16d3b81f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:18.765013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139b6ad-c173-483f-ac99-bfb47aa4015f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:18.771207 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 467/923] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Sep 6 21:01:18 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:01:18.773535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bede435-47d0-46c1-89ff-468f16d3b81f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:18.804228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea1eef-6edc-4b77-88e3-19fc5fd042eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:18.804989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea1eef-6edc-4b77-88e3-19fc5fd042eb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:18.805265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea1eef-6edc-4b77-88e3-19fc5fd042eb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:18.805530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea1eef-6edc-4b77-88e3-19fc5fd042eb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:18.806123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea1eef-6edc-4b77-88e3-19fc5fd042eb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:18.807542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea1eef-6edc-4b77-88e3-19fc5fd042eb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-10ea1eef-6edc-4b77-88e3-19fc5fd042eb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:18.818210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:18.819305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 21:01:18.820218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:18.824587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 457/924] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:01:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:18.825596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea1eef-6edc-4b77-88e3-19fc5fd042eb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:18.907595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:18.913276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bede435-47d0-46c1-89ff-468f16d3b81f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:18.955935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10e5b2-4998-4e2f-846b-53acabb8bb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:18.957292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10e5b2-4998-4e2f-846b-53acabb8bb07 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:18.957790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10e5b2-4998-4e2f-846b-53acabb8bb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:18.958264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10e5b2-4998-4e2f-846b-53acabb8bb07 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:18.967484 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea1eef-6edc-4b77-88e3-19fc5fd042eb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:18.971919 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10e5b2-4998-4e2f-846b-53acabb8bb07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5c10e5b2-4998-4e2f-846b-53acabb8bb07 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:18.977400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:18.978365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:18.979227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:18.979227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:18.979227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:18.979227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:18.979227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:18.979227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:18.979227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:18.979227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:18.979227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:18.979227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:18.979227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 21:01:18.979227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:18.981652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 468/925] 10.222.0.79 () {56 vars in 1120 bytes} [Tue Sep 6 21:01:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:18.988508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10e5b2-4998-4e2f-846b-53acabb8bb07 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:19.013260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ad5b5-b371-43a6-977d-f32e87d878c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:19.013836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ad5b5-b371-43a6-977d-f32e87d878c3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:19.014112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ad5b5-b371-43a6-977d-f32e87d878c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:19.014376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ad5b5-b371-43a6-977d-f32e87d878c3 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:19.015013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ad5b5-b371-43a6-977d-f32e87d878c3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:19.015312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ad5b5-b371-43a6-977d-f32e87d878c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:19.016751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ad5b5-b371-43a6-977d-f32e87d878c3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:19.023046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 458/926] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:01:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:19.028964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ad5b5-b371-43a6-977d-f32e87d878c3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:19.034933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:19.110916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10e5b2-4998-4e2f-846b-53acabb8bb07 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:19.143976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ad5b5-b371-43a6-977d-f32e87d878c3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:19.151960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab67429-658a-4f5b-8340-716f8dc9802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:19.152616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab67429-658a-4f5b-8340-716f8dc9802a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:19.152914 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab67429-658a-4f5b-8340-716f8dc9802a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:19.153186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab67429-658a-4f5b-8340-716f8dc9802a None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:19.153861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab67429-658a-4f5b-8340-716f8dc9802a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:19.154160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab67429-658a-4f5b-8340-716f8dc9802a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:19.155539 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab67429-658a-4f5b-8340-716f8dc9802a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:19.163760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 469/927] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Sep 6 21:01:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:19.170510 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:19.171987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31af0c3-bcf2-4441-8916-da59a637a7a4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:19.172532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31af0c3-bcf2-4441-8916-da59a637a7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:19.173208 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31af0c3-bcf2-4441-8916-da59a637a7a4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:19.175405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 470/928] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:19 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:19.176463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31af0c3-bcf2-4441-8916-da59a637a7a4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:19.181656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851ec5b-67aa-42a6-9412-921c1902990d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:19.182265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ec5b-67aa-42a6-9412-921c1902990d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:19.182538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ec5b-67aa-42a6-9412-921c1902990d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:19.182805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ec5b-67aa-42a6-9412-921c1902990d None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:19.183434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851ec5b-67aa-42a6-9412-921c1902990d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:19.183759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851ec5b-67aa-42a6-9412-921c1902990d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:19.185649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851ec5b-67aa-42a6-9412-921c1902990d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:19.191037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 459/929] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Sep 6 21:01:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:19.192111 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88335599-fd40-40ad-b40d-54c3dfc6c1fa None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:19.192392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88335599-fd40-40ad-b40d-54c3dfc6c1fa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:19.192675 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88335599-fd40-40ad-b40d-54c3dfc6c1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:19.211248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-88335599-fd40-40ad-b40d-54c3dfc6c1fa None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:19.212434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-88335599-fd40-40ad-b40d-54c3dfc6c1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:19.289848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:19.290287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 460/930] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:19.291205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88335599-fd40-40ad-b40d-54c3dfc6c1fa None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:19.291640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31af0c3-bcf2-4441-8916-da59a637a7a4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:19.338169 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027a1e4-2eae-40c6-9d6e-7b299625d967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:19.339343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027a1e4-2eae-40c6-9d6e-7b299625d967 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:19.339343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027a1e4-2eae-40c6-9d6e-7b299625d967 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:19.339343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027a1e4-2eae-40c6-9d6e-7b299625d967 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:19.351033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027a1e4-2eae-40c6-9d6e-7b299625d967 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:19.352637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027a1e4-2eae-40c6-9d6e-7b299625d967 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1027a1e4-2eae-40c6-9d6e-7b299625d967 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:19.367735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:19.369078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:19.369946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:19.369946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:19.369946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:19.369946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 471/931] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:19.374744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027a1e4-2eae-40c6-9d6e-7b299625d967 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:19.420087 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88335599-fd40-40ad-b40d-54c3dfc6c1fa None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:19.441692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:19.465977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38436ab2-7001-439d-af20-51d8dd951d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:19.466599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38436ab2-7001-439d-af20-51d8dd951d80 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:19.466878 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38436ab2-7001-439d-af20-51d8dd951d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:19.467308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38436ab2-7001-439d-af20-51d8dd951d80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:19.468077 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38436ab2-7001-439d-af20-51d8dd951d80 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:19.469763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38436ab2-7001-439d-af20-51d8dd951d80 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:19.491321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027a1e4-2eae-40c6-9d6e-7b299625d967 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:19.503284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 461/932] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:01:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:19.546571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442fb3c-6a0c-4ca8-847d-aeca87647309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:19.547439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442fb3c-6a0c-4ca8-847d-aeca87647309 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:19.547864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442fb3c-6a0c-4ca8-847d-aeca87647309 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:19.548330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442fb3c-6a0c-4ca8-847d-aeca87647309 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:19.549189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442fb3c-6a0c-4ca8-847d-aeca87647309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:19.549694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442fb3c-6a0c-4ca8-847d-aeca87647309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:19.551407 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442fb3c-6a0c-4ca8-847d-aeca87647309 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:19.568160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 472/933] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:19.573988 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38436ab2-7001-439d-af20-51d8dd951d80 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:19.574673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442fb3c-6a0c-4ca8-847d-aeca87647309 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:19.579791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:19.698656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442fb3c-6a0c-4ca8-847d-aeca87647309 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:19.737170 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90807f-2d96-4e65-973a-4022ac32124b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:19.737834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90807f-2d96-4e65-973a-4022ac32124b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:19.738085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90807f-2d96-4e65-973a-4022ac32124b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:19.738354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90807f-2d96-4e65-973a-4022ac32124b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:19.738954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90807f-2d96-4e65-973a-4022ac32124b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:19.740340 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90807f-2d96-4e65-973a-4022ac32124b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:19.748220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38436ab2-7001-439d-af20-51d8dd951d80 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2c90807f-2d96-4e65-973a-4022ac32124b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:19.750237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:19.751342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 21:01:19.752308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:19.753110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 473/934] 10.222.0.79 () {56 vars in 1132 bytes} [Tue Sep 6 21:01:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:19.760186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90807f-2d96-4e65-973a-4022ac32124b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:19.815572 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ca43f-4568-4942-a77b-037ff487333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:19.816191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ca43f-4568-4942-a77b-037ff487333a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:19.816475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ca43f-4568-4942-a77b-037ff487333a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:19.816788 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ca43f-4568-4942-a77b-037ff487333a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:19.828390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ca43f-4568-4942-a77b-037ff487333a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:19.830300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ca43f-4568-4942-a77b-037ff487333a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-eb8ca43f-4568-4942-a77b-037ff487333a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:01:19.846387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:19.847415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:19.848327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:19.848327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:19.848327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:01:19.848327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:01:19.848327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:01:19.848327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:01:19.848327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:01:19.848327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:01:19.848327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 21:01:19.848327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:19.848874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:19.852828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 462/935] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Sep 6 21:01:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:19.855647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ca43f-4568-4942-a77b-037ff487333a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:19.890801 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90807f-2d96-4e65-973a-4022ac32124b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:19.927397 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4231de3-3253-4ca6-9ad1-772b603e60ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:19.928079 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4231de3-3253-4ca6-9ad1-772b603e60ba None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:19.928329 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:19.928434 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4231de3-3253-4ca6-9ad1-772b603e60ba None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:19.928628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4231de3-3253-4ca6-9ad1-772b603e60ba None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:19.929349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4231de3-3253-4ca6-9ad1-772b603e60ba None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:19.929663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4231de3-3253-4ca6-9ad1-772b603e60ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:19.931600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4231de3-3253-4ca6-9ad1-772b603e60ba None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:19.942638 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 474/936] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Sep 6 21:01:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:19.943967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4231de3-3253-4ca6-9ad1-772b603e60ba None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:19.989063 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ca43f-4568-4942-a77b-037ff487333a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:20.065459 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4231de3-3253-4ca6-9ad1-772b603e60ba None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:20.069166 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2721df2-fcbe-4988-bb44-834c57e048de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:20.069745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2721df2-fcbe-4988-bb44-834c57e048de None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:20.070022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2721df2-fcbe-4988-bb44-834c57e048de None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:20.070290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2721df2-fcbe-4988-bb44-834c57e048de None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:20.071178 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2721df2-fcbe-4988-bb44-834c57e048de None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:20.071489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2721df2-fcbe-4988-bb44-834c57e048de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:20.072971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2721df2-fcbe-4988-bb44-834c57e048de None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:20.091985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 463/937] 10.222.0.79 () {56 vars in 1149 bytes} [Tue Sep 6 21:01:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:20.099173 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2721df2-fcbe-4988-bb44-834c57e048de None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:20.109798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:20.121877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde33952-eb05-4b4b-9633-d310b6e58ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:20.122928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33952-eb05-4b4b-9633-d310b6e58ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:20.123204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33952-eb05-4b4b-9633-d310b6e58ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:20.123471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33952-eb05-4b4b-9633-d310b6e58ab9 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:20.124108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde33952-eb05-4b4b-9633-d310b6e58ab9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:20.124408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde33952-eb05-4b4b-9633-d310b6e58ab9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:20.126472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde33952-eb05-4b4b-9633-d310b6e58ab9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:20.134181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 475/938] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Sep 6 21:01:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:20.141657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde33952-eb05-4b4b-9633-d310b6e58ab9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:20.203768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:20.213112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2721df2-fcbe-4988-bb44-834c57e048de None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:20.250093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654bec9-ebc2-4051-8d02-e868897a9b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:20.250733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654bec9-ebc2-4051-8d02-e868897a9b36 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:20.251014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654bec9-ebc2-4051-8d02-e868897a9b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:20.251282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654bec9-ebc2-4051-8d02-e868897a9b36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:20.262513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654bec9-ebc2-4051-8d02-e868897a9b36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:20.263957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654bec9-ebc2-4051-8d02-e868897a9b36 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:20.268189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde33952-eb05-4b4b-9633-d310b6e58ab9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d654bec9-ebc2-4051-8d02-e868897a9b36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:20.287705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:20.288922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:20.289946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:20.289946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:20.289946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:20.292369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 464/939] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:20.301459 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654bec9-ebc2-4051-8d02-e868897a9b36 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:20.323775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c134957-831f-46b3-818b-513f175ee51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:20.324515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c134957-831f-46b3-818b-513f175ee51c None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:20.324800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c134957-831f-46b3-818b-513f175ee51c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:20.325161 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c134957-831f-46b3-818b-513f175ee51c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:20.326021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c134957-831f-46b3-818b-513f175ee51c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:20.328923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c134957-831f-46b3-818b-513f175ee51c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:20.365273 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 476/940] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:01:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:20.436120 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654bec9-ebc2-4051-8d02-e868897a9b36 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:20.480679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc85f6e-d089-4770-af7e-58c61982e4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:20.481309 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc85f6e-d089-4770-af7e-58c61982e4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:20.481588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc85f6e-d089-4770-af7e-58c61982e4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:20.481861 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc85f6e-d089-4770-af7e-58c61982e4e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:20.482518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc85f6e-d089-4770-af7e-58c61982e4e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:20.482822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc85f6e-d089-4770-af7e-58c61982e4e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:20.484224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc85f6e-d089-4770-af7e-58c61982e4e8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:20.495467 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:20.505844 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c134957-831f-46b3-818b-513f175ee51c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:20.506355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 465/941] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:20.575121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:20.629986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c134957-831f-46b3-818b-513f175ee51c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:20.670148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc31d8-ad56-4cdd-9179-4ced79daf198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:20.670803 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc31d8-ad56-4cdd-9179-4ced79daf198 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:20.671083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc31d8-ad56-4cdd-9179-4ced79daf198 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:20.671355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc31d8-ad56-4cdd-9179-4ced79daf198 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:20.679356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc31d8-ad56-4cdd-9179-4ced79daf198 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:20.680883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc31d8-ad56-4cdd-9179-4ced79daf198 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-36fc31d8-ad56-4cdd-9179-4ced79daf198 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:01:20.694171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:20.695112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:20.696065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:01:20.696065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:01:20.696065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:01:20.696065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:01:20.696065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:01:20.696065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:01:20.696065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:20.696065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:20.696065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 477/942] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:20.702881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc85f6e-d089-4770-af7e-58c61982e4e8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:20.813083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc85f6e-d089-4770-af7e-58c61982e4e8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:20.848210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14342d6e-3776-4760-9c3b-1a5a618d39c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:20.848768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14342d6e-3776-4760-9c3b-1a5a618d39c1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:20.849045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14342d6e-3776-4760-9c3b-1a5a618d39c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:20.849313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14342d6e-3776-4760-9c3b-1a5a618d39c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:20.850014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14342d6e-3776-4760-9c3b-1a5a618d39c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:20.850362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14342d6e-3776-4760-9c3b-1a5a618d39c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:20.851926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14342d6e-3776-4760-9c3b-1a5a618d39c1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:20.864618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 466/943] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:01:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:20.870321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc31d8-ad56-4cdd-9179-4ced79daf198 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:20.991267 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc31d8-ad56-4cdd-9179-4ced79daf198 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:21.021390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:21.027117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605202d-d158-45dc-b1c1-4e914393ae13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:21.028068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605202d-d158-45dc-b1c1-4e914393ae13 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:21.028407 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605202d-d158-45dc-b1c1-4e914393ae13 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:21.028877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605202d-d158-45dc-b1c1-4e914393ae13 None admin] PATH_INFO: `/v3/projects/05af249ba0a047fe83aefb078e203e53/users/c09b4900d97b4b35b617af9a42baf9ee/roles/6bc0365fc21845598826c2848897e72b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:21.031336 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605202d-d158-45dc-b1c1-4e914393ae13 None admin] RBAC: Authorizing `identity:create_grant(project_id=05af249ba0a047fe83aefb078e203e53, user_id=c09b4900d97b4b35b617af9a42baf9ee, role_id=6bc0365fc21845598826c2848897e72b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:21.032852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605202d-d158-45dc-b1c1-4e914393ae13 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:21.047495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 478/944] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:01:20 2022] PUT /identity/v3/projects/05af249ba0a047fe83aefb078e203e53/users/c09b4900d97b4b35b617af9a42baf9ee/roles/6bc0365fc21845598826c2848897e72b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:01:21.186829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7bf49-f3e3-48d4-bcb1-3f9dc767b1f1 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:21.187010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7bf49-f3e3-48d4-bcb1-3f9dc767b1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:21.187279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7bf49-f3e3-48d4-bcb1-3f9dc767b1f1 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:21.189071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 467/945] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:21.195067 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de622e-a509-4ecf-9894-dc678522e541 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:21.195365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de622e-a509-4ecf-9894-dc678522e541 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:21.195610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de622e-a509-4ecf-9894-dc678522e541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:21.215771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a9de622e-a509-4ecf-9894-dc678522e541 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:21.217016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a9de622e-a509-4ecf-9894-dc678522e541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:21.295382 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 479/946] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:21.561150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7bf49-f3e3-48d4-bcb1-3f9dc767b1f1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:21.626027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:21.670628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7bf49-f3e3-48d4-bcb1-3f9dc767b1f1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:21.707422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb1fbf-196d-4ebd-9960-5503973117e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:21.708108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb1fbf-196d-4ebd-9960-5503973117e3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:21.709974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb1fbf-196d-4ebd-9960-5503973117e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:21.709974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb1fbf-196d-4ebd-9960-5503973117e3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:21.709974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb1fbf-196d-4ebd-9960-5503973117e3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:21.710602 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb1fbf-196d-4ebd-9960-5503973117e3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-44fb1fbf-196d-4ebd-9960-5503973117e3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:21.720005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:21.721074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:21.724702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 468/947] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Sep 6 21:01:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:21.728497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de622e-a509-4ecf-9894-dc678522e541 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:21.841271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de622e-a509-4ecf-9894-dc678522e541 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:21.842826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:21.878179 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffe42f-8ff6-42b2-be56-be0f5c9b5bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:21.878860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffe42f-8ff6-42b2-be56-be0f5c9b5bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:21.879110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffe42f-8ff6-42b2-be56-be0f5c9b5bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:21.879350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffe42f-8ff6-42b2-be56-be0f5c9b5bb3 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:21.879995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffe42f-8ff6-42b2-be56-be0f5c9b5bb3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:21.880299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffe42f-8ff6-42b2-be56-be0f5c9b5bb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:21.881736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffe42f-8ff6-42b2-be56-be0f5c9b5bb3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:21.887413 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 480/948] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Sep 6 21:01:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:21.893103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb1fbf-196d-4ebd-9960-5503973117e3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:21.930532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46062c6-ae72-4bd2-a3eb-1f1cac655423 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:21.930859 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46062c6-ae72-4bd2-a3eb-1f1cac655423 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:21.931085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46062c6-ae72-4bd2-a3eb-1f1cac655423 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:21.932586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 481/949] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:21.939535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe3eb8-0cee-4576-bb46-763b48bb0cf4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:21.939816 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe3eb8-0cee-4576-bb46-763b48bb0cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:21.940091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe3eb8-0cee-4576-bb46-763b48bb0cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:21.963142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-74fe3eb8-0cee-4576-bb46-763b48bb0cf4 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:21.964707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-74fe3eb8-0cee-4576-bb46-763b48bb0cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:22.013961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb1fbf-196d-4ebd-9960-5503973117e3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:22.052847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 482/950] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:22.053100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:22.068692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf41355-fd32-40d4-8b5d-7483e447e0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:22.069652 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41355-fd32-40d4-8b5d-7483e447e0fb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:22.070242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41355-fd32-40d4-8b5d-7483e447e0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:22.070797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41355-fd32-40d4-8b5d-7483e447e0fb None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:22.071769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf41355-fd32-40d4-8b5d-7483e447e0fb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:22.072406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf41355-fd32-40d4-8b5d-7483e447e0fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:22.074430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf41355-fd32-40d4-8b5d-7483e447e0fb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:22.081330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 469/951] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Sep 6 21:01:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:22.088287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe3eb8-0cee-4576-bb46-763b48bb0cf4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:22.192725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:22.201703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe3eb8-0cee-4576-bb46-763b48bb0cf4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:22.239148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b10ba4-3268-41ec-b208-6c426eef4cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:22.240477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b10ba4-3268-41ec-b208-6c426eef4cfa None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:22.241131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b10ba4-3268-41ec-b208-6c426eef4cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:22.241607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b10ba4-3268-41ec-b208-6c426eef4cfa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:22.242926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b10ba4-3268-41ec-b208-6c426eef4cfa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:22.244889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b10ba4-3268-41ec-b208-6c426eef4cfa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:22.269378 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 483/952] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:01:22 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:22.358958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf41355-fd32-40d4-8b5d-7483e447e0fb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:22.496810 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf41355-fd32-40d4-8b5d-7483e447e0fb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:22.546409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b93d14-fac4-47e1-81d5-09b5b1c7e6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:22.547381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b93d14-fac4-47e1-81d5-09b5b1c7e6fa None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:22.547855 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b93d14-fac4-47e1-81d5-09b5b1c7e6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:22.548344 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b93d14-fac4-47e1-81d5-09b5b1c7e6fa None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:22.549228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b93d14-fac4-47e1-81d5-09b5b1c7e6fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:22.551010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b93d14-fac4-47e1-81d5-09b5b1c7e6fa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:22.553899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: WARNING keystone.server.flask.common [None req-f5b93d14-fac4-47e1-81d5-09b5b1c7e6fa None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 06 21:01:22.574506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.sql.core [None req-f5b93d14-fac4-47e1-81d5-09b5b1c7e6fa None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 21:01:22.574506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 21:01:22.574506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [parameters: {'id': '12465a42129c482daa0421f3e6ea10eb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 21:01:22.574506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81697) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 21:01:22.583021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f5b93d14-fac4-47e1-81d5-09b5b1c7e6fa None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 12465a42129c482daa0421f3e6ea10eb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 12465a42129c482daa0421f3e6ea10eb. Sep 06 21:01:22.583021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:22.583021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 21:01:22.583021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 21:01:22.583021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 21:01:22.583021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 21:01:22.583021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 21:01:22.583021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 21:01:22.583021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 21:01:22.583021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Sep 06 21:01:22.583021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 21:01:22.583021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 21:01:22.583021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 06 21:01:22.583877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application yield resource Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise self.value Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application session.commit() Sep 06 21:01:22.584928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.flush() Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application compat.raise_( Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 21:01:22.585812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 21:01:22.586729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 21:01:22.587513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [parameters: {'id': '12465a42129c482daa0421f3e6ea10eb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:22.588370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:22.591162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:22.594031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:22.594031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 06 21:01:22.594031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 06 21:01:22.594031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:22.594031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:22.594031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 06 21:01:22.594031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 06 21:01:22.594031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 12465a42129c482daa0421f3e6ea10eb. Sep 06 21:01:22.594031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:22.597521 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 470/953] 10.222.0.79 () {60 vars in 1140 bytes} [Tue Sep 6 21:01:22 2022] POST /identity/v3/projects => generated 262 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:22.608892 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b10ba4-3268-41ec-b208-6c426eef4cfa None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:22.725534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b10ba4-3268-41ec-b208-6c426eef4cfa None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:22.763360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd88e9-7056-49d7-be3a-5a1b97377b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:22.764408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd88e9-7056-49d7-be3a-5a1b97377b8b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:22.764956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd88e9-7056-49d7-be3a-5a1b97377b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:22.765445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd88e9-7056-49d7-be3a-5a1b97377b8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:22.773642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd88e9-7056-49d7-be3a-5a1b97377b8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:22.775477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd88e9-7056-49d7-be3a-5a1b97377b8b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:22.779568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32d3d1-63a9-495d-84a0-530388c29ddc None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:22.780077 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32d3d1-63a9-495d-84a0-530388c29ddc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:22.780156 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32d3d1-63a9-495d-84a0-530388c29ddc None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a1dd88e9-7056-49d7-be3a-5a1b97377b8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:01:22.789788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:22.790763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:22.791731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:01:22.791731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:01:22.791731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:01:22.791731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:01:22.791731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:01:22.791731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:01:22.791731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:22.791731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:22.792167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 471/954] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:22.792167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda9449-41d5-4415-b38c-a870b8414e59 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:22.792167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda9449-41d5-4415-b38c-a870b8414e59 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:22.792344 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda9449-41d5-4415-b38c-a870b8414e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:22.794656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 484/955] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:22.801026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd88e9-7056-49d7-be3a-5a1b97377b8b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:22.815644 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5bda9449-41d5-4415-b38c-a870b8414e59 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:22.817498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5bda9449-41d5-4415-b38c-a870b8414e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:22.817962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:22.856791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:22.901047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 472/956] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:22.916009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd88e9-7056-49d7-be3a-5a1b97377b8b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:22.951801 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33adf44e-5bd9-416e-add3-21e2c4ad1294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:22.952615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33adf44e-5bd9-416e-add3-21e2c4ad1294 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:22.952989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33adf44e-5bd9-416e-add3-21e2c4ad1294 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:22.953323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33adf44e-5bd9-416e-add3-21e2c4ad1294 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:22.954065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33adf44e-5bd9-416e-add3-21e2c4ad1294 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:22.954429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33adf44e-5bd9-416e-add3-21e2c4ad1294 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:22.956051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33adf44e-5bd9-416e-add3-21e2c4ad1294 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:22.980841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 485/957] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Sep 6 21:01:22 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:23.166661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda9449-41d5-4415-b38c-a870b8414e59 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:23.282013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda9449-41d5-4415-b38c-a870b8414e59 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:23.320768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e39417-1560-4b3c-8fbb-1bc57210dd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:23.321664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e39417-1560-4b3c-8fbb-1bc57210dd86 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:23.322121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e39417-1560-4b3c-8fbb-1bc57210dd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:23.322553 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e39417-1560-4b3c-8fbb-1bc57210dd86 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:23.323283 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e39417-1560-4b3c-8fbb-1bc57210dd86 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:23.324995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e39417-1560-4b3c-8fbb-1bc57210dd86 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-73e39417-1560-4b3c-8fbb-1bc57210dd86 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:23.334497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:23.335794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 21:01:23.336973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:23.339554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 473/958] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:23.345740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33adf44e-5bd9-416e-add3-21e2c4ad1294 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:23.387987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e140b-58fc-4265-b943-5fea38fbc78a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:23.388252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e140b-58fc-4265-b943-5fea38fbc78a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:23.388530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e140b-58fc-4265-b943-5fea38fbc78a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:23.390208 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 474/959] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:01:23.395395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fa205-e38c-4526-adf3-3eac62682089 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:23.395665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fa205-e38c-4526-adf3-3eac62682089 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:23.395938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fa205-e38c-4526-adf3-3eac62682089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:23.416267 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-815fa205-e38c-4526-adf3-3eac62682089 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:23.417737 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-815fa205-e38c-4526-adf3-3eac62682089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:23.424920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:23.452202 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:23.477282 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33adf44e-5bd9-416e-add3-21e2c4ad1294 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:23.533111 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 475/960] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:23.543618 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6afbf-ad69-4058-bf98-fc38fc729e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:23.545067 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6afbf-ad69-4058-bf98-fc38fc729e19 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:23.545766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6afbf-ad69-4058-bf98-fc38fc729e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:23.546399 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6afbf-ad69-4058-bf98-fc38fc729e19 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:23.547757 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6afbf-ad69-4058-bf98-fc38fc729e19 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:23.548479 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6afbf-ad69-4058-bf98-fc38fc729e19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:23.551236 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6afbf-ad69-4058-bf98-fc38fc729e19 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:23.561644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 486/961] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Sep 6 21:01:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:23.570696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fa205-e38c-4526-adf3-3eac62682089 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:23.760979 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fa205-e38c-4526-adf3-3eac62682089 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:23.821533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b243b-4863-44c3-9c20-03832fb02964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:23.823611 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b243b-4863-44c3-9c20-03832fb02964 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:23.824301 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b243b-4863-44c3-9c20-03832fb02964 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:23.825146 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b243b-4863-44c3-9c20-03832fb02964 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:23.826432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b243b-4863-44c3-9c20-03832fb02964 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:23.827072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b243b-4863-44c3-9c20-03832fb02964 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:23.829843 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b243b-4863-44c3-9c20-03832fb02964 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:23.838815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 476/962] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Sep 6 21:01:23 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:24.009502 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6afbf-ad69-4058-bf98-fc38fc729e19 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:24.129475 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6afbf-ad69-4058-bf98-fc38fc729e19 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:24.165814 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8204d5-578d-4efc-9280-2d5922967837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:24.166522 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8204d5-578d-4efc-9280-2d5922967837 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:24.166803 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8204d5-578d-4efc-9280-2d5922967837 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:24.167073 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8204d5-578d-4efc-9280-2d5922967837 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:24.169837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:24.178819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8204d5-578d-4efc-9280-2d5922967837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2e8204d5-578d-4efc-9280-2d5922967837 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:24.183681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:24.184741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:24.185958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:24.185958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:24.185958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:24.185958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:24.185958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:24.185958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:24.185958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:24.185958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:24.185958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:24.185958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:24.185958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 21:01:24.185958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:24.185958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 487/963] 10.222.0.79 () {56 vars in 1120 bytes} [Tue Sep 6 21:01:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:24.191539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b243b-4863-44c3-9c20-03832fb02964 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:24.223034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:24.303955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b243b-4863-44c3-9c20-03832fb02964 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:24.343336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33db4fd-e87e-4bea-946c-43a8c3ae3df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:24.344354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33db4fd-e87e-4bea-946c-43a8c3ae3df0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:24.344835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33db4fd-e87e-4bea-946c-43a8c3ae3df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:24.345249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33db4fd-e87e-4bea-946c-43a8c3ae3df0 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:24.346079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33db4fd-e87e-4bea-946c-43a8c3ae3df0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:24.346526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33db4fd-e87e-4bea-946c-43a8c3ae3df0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:24.348073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33db4fd-e87e-4bea-946c-43a8c3ae3df0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:24.358322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 477/964] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Sep 6 21:01:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:24.363865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8204d5-578d-4efc-9280-2d5922967837 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:24.468845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8204d5-578d-4efc-9280-2d5922967837 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:24.507067 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c805b-36f8-4edb-a4fe-5206a2027321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:24.507729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c805b-36f8-4edb-a4fe-5206a2027321 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:24.508004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c805b-36f8-4edb-a4fe-5206a2027321 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:24.508322 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c805b-36f8-4edb-a4fe-5206a2027321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:24.520935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c805b-36f8-4edb-a4fe-5206a2027321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:24.522424 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c805b-36f8-4edb-a4fe-5206a2027321 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-452c805b-36f8-4edb-a4fe-5206a2027321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:24.538316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:24.539694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:24.541788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:24.541788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:24.541788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:24.541788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 488/965] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:24.545628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33db4fd-e87e-4bea-946c-43a8c3ae3df0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:24.627624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367d62f-6cd7-4d24-8916-43037b1e3d62 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:24.627904 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367d62f-6cd7-4d24-8916-43037b1e3d62 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:24.628176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367d62f-6cd7-4d24-8916-43037b1e3d62 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:24.629805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 489/966] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:01:24.635044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925716f4-2af8-43a7-9169-f48d1f76a868 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:24.635321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925716f4-2af8-43a7-9169-f48d1f76a868 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:24.635586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925716f4-2af8-43a7-9169-f48d1f76a868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:24.655351 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-925716f4-2af8-43a7-9169-f48d1f76a868 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:24.656755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-925716f4-2af8-43a7-9169-f48d1f76a868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:24.662711 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:24.674106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33db4fd-e87e-4bea-946c-43a8c3ae3df0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:24.716640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f124b2a-e140-4c52-8ac0-84d49a79a5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:24.717382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f124b2a-e140-4c52-8ac0-84d49a79a5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:24.717799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f124b2a-e140-4c52-8ac0-84d49a79a5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:24.718118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f124b2a-e140-4c52-8ac0-84d49a79a5f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:24.718782 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f124b2a-e140-4c52-8ac0-84d49a79a5f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:24.719145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f124b2a-e140-4c52-8ac0-84d49a79a5f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:24.720487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f124b2a-e140-4c52-8ac0-84d49a79a5f3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:24.722898 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:24.738253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 478/967] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:24.743449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f124b2a-e140-4c52-8ac0-84d49a79a5f3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:24.772706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 490/968] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:24.861863 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f124b2a-e140-4c52-8ac0-84d49a79a5f3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:24.901194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80972547-e255-4163-9731-ea2d0b40eb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:24.901873 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80972547-e255-4163-9731-ea2d0b40eb09 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:24.902153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80972547-e255-4163-9731-ea2d0b40eb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:24.902456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80972547-e255-4163-9731-ea2d0b40eb09 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:24.911619 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80972547-e255-4163-9731-ea2d0b40eb09 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:24.913177 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80972547-e255-4163-9731-ea2d0b40eb09 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-80972547-e255-4163-9731-ea2d0b40eb09 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:01:24.926877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:24.927980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:24.928951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:24.928951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:24.928951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:01:24.928951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:01:24.928951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:01:24.928951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:01:24.928951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:01:24.928951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:01:24.928951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 21:01:24.928951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:24.928951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 479/969] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Sep 6 21:01:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:24.945840 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925716f4-2af8-43a7-9169-f48d1f76a868 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:25.066065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925716f4-2af8-43a7-9169-f48d1f76a868 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:25.087863 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80972547-e255-4163-9731-ea2d0b40eb09 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:25.102449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:25.120369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10eed4-7bb5-4785-8c5a-4ec6dcafab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:25.121849 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10eed4-7bb5-4785-8c5a-4ec6dcafab91 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:25.122338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10eed4-7bb5-4785-8c5a-4ec6dcafab91 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:25.122932 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10eed4-7bb5-4785-8c5a-4ec6dcafab91 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:25.123967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10eed4-7bb5-4785-8c5a-4ec6dcafab91 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:25.129123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10eed4-7bb5-4785-8c5a-4ec6dcafab91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:25.131496 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10eed4-7bb5-4785-8c5a-4ec6dcafab91 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:25.166969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 491/970] 10.222.0.79 () {56 vars in 1149 bytes} [Tue Sep 6 21:01:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:25.178330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10eed4-7bb5-4785-8c5a-4ec6dcafab91 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:25.212402 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80972547-e255-4163-9731-ea2d0b40eb09 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:25.245419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:25.257072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9fac76-686d-4f63-9bdb-4545e8006294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:25.257741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fac76-686d-4f63-9bdb-4545e8006294 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:25.258024 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fac76-686d-4f63-9bdb-4545e8006294 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:25.258291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fac76-686d-4f63-9bdb-4545e8006294 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:25.258894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9fac76-686d-4f63-9bdb-4545e8006294 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:25.260425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9fac76-686d-4f63-9bdb-4545e8006294 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-bf9fac76-686d-4f63-9bdb-4545e8006294 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:25.272135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:25.273260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:25.274137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 480/971] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Sep 6 21:01:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:25.286119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9fac76-686d-4f63-9bdb-4545e8006294 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:25.361649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10eed4-7bb5-4785-8c5a-4ec6dcafab91 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:25.398930 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717584e-c209-42ef-b170-4121bd2326c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:25.399615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717584e-c209-42ef-b170-4121bd2326c0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:25.399994 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717584e-c209-42ef-b170-4121bd2326c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:25.400163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717584e-c209-42ef-b170-4121bd2326c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:25.421526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:25.460439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9fac76-686d-4f63-9bdb-4545e8006294 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:25.512336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f8e63-903d-4353-98f8-24197510a95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:25.512336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f8e63-903d-4353-98f8-24197510a95f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:25.512336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f8e63-903d-4353-98f8-24197510a95f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:25.512784 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f8e63-903d-4353-98f8-24197510a95f None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:25.513242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f8e63-903d-4353-98f8-24197510a95f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:25.513546 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f8e63-903d-4353-98f8-24197510a95f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:25.514929 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f8e63-903d-4353-98f8-24197510a95f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:25.517823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717584e-c209-42ef-b170-4121bd2326c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:25.518741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717584e-c209-42ef-b170-4121bd2326c0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:25.520128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 481/972] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Sep 6 21:01:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:25.521143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc24c1-5684-4a12-8402-4c9b8e0ad857 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:25.521425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc24c1-5684-4a12-8402-4c9b8e0ad857 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:25.522389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc24c1-5684-4a12-8402-4c9b8e0ad857 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:25.527812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 482/973] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:25.527812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc24c1-5684-4a12-8402-4c9b8e0ad857 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:25.558941 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:25.644730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 492/974] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:01:25 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 21:01:25.646332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357d30f-40d6-4d80-b89e-3a53db6e8fd2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:25.646819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357d30f-40d6-4d80-b89e-3a53db6e8fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:25.647279 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357d30f-40d6-4d80-b89e-3a53db6e8fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:25.650467 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc24c1-5684-4a12-8402-4c9b8e0ad857 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:25.666727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a357d30f-40d6-4d80-b89e-3a53db6e8fd2 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:25.667944 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a357d30f-40d6-4d80-b89e-3a53db6e8fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:25.697301 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1d5b1-5a40-40f3-ab05-4005464dd18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:25.697968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1d5b1-5a40-40f3-ab05-4005464dd18e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:25.698205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1d5b1-5a40-40f3-ab05-4005464dd18e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:25.698476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1d5b1-5a40-40f3-ab05-4005464dd18e None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:25.699253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1d5b1-5a40-40f3-ab05-4005464dd18e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:25.699557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1d5b1-5a40-40f3-ab05-4005464dd18e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:25.701171 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1d5b1-5a40-40f3-ab05-4005464dd18e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:25.707473 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 483/975] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Sep 6 21:01:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:25.708287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1d5b1-5a40-40f3-ab05-4005464dd18e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:25.711196 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:25.756494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 493/976] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:25.756494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357d30f-40d6-4d80-b89e-3a53db6e8fd2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:25.837781 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:25.847544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1d5b1-5a40-40f3-ab05-4005464dd18e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:25.880603 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357d30f-40d6-4d80-b89e-3a53db6e8fd2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:25.900967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c634416-c6fd-4aff-9b68-28151506ffc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:25.901494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c634416-c6fd-4aff-9b68-28151506ffc5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:25.901861 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c634416-c6fd-4aff-9b68-28151506ffc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:25.902375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c634416-c6fd-4aff-9b68-28151506ffc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:25.914627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c634416-c6fd-4aff-9b68-28151506ffc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:25.916183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c634416-c6fd-4aff-9b68-28151506ffc5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:25.920757 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d08f8f-ef90-4040-b7e0-159921e86684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:25.921591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d08f8f-ef90-4040-b7e0-159921e86684 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:25.922042 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d08f8f-ef90-4040-b7e0-159921e86684 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:25.922482 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d08f8f-ef90-4040-b7e0-159921e86684 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:25.923797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d08f8f-ef90-4040-b7e0-159921e86684 None admin] RBAC: Adding query filter params (interface=admin, service_id=56563e6b9bae4ead93a8958fa2f9af5c, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:25.924284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d08f8f-ef90-4040-b7e0-159921e86684 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:25.926523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d08f8f-ef90-4040-b7e0-159921e86684 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2c634416-c6fd-4aff-9b68-28151506ffc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:25.931976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:25.932971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:25.933898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:25.933898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:25.933898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:25.935572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 494/977] 10.222.0.79 () {56 vars in 1249 bytes} [Tue Sep 6 21:01:25 2022] GET /identity/v3/endpoints?service_id=56563e6b9bae4ead93a8958fa2f9af5c&interface=admin®ion_id=RegionOne => generated 193 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:25.938184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 484/978] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:25.939250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d08f8f-ef90-4040-b7e0-159921e86684 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:25.940638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c634416-c6fd-4aff-9b68-28151506ffc5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:26.007459 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:26.054830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d08f8f-ef90-4040-b7e0-159921e86684 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:26.061359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c634416-c6fd-4aff-9b68-28151506ffc5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:26.088089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:26.096154 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65af595-7ffb-451d-8732-f5b8911457f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:26.097110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65af595-7ffb-451d-8732-f5b8911457f3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:26.097110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65af595-7ffb-451d-8732-f5b8911457f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:26.097674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65af595-7ffb-451d-8732-f5b8911457f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:26.098008 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65af595-7ffb-451d-8732-f5b8911457f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:26.098312 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65af595-7ffb-451d-8732-f5b8911457f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:26.099707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65af595-7ffb-451d-8732-f5b8911457f3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:26.100099 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a99a1-8ae1-43b5-acc3-f758b1367fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:26.100750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a99a1-8ae1-43b5-acc3-f758b1367fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:26.101019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a99a1-8ae1-43b5-acc3-f758b1367fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:26.101314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a99a1-8ae1-43b5-acc3-f758b1367fc4 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:26.101963 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a99a1-8ae1-43b5-acc3-f758b1367fc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:26.103351 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a99a1-8ae1-43b5-acc3-f758b1367fc4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:26.110393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 485/979] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Sep 6 21:01:25 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:01:26.110972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a99a1-8ae1-43b5-acc3-f758b1367fc4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:26.122053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 495/980] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:26.123902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65af595-7ffb-451d-8732-f5b8911457f3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:26.265637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a99a1-8ae1-43b5-acc3-f758b1367fc4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:26.282116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65af595-7ffb-451d-8732-f5b8911457f3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:26.285967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:26.310847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e983d-88ed-4fcd-b7ec-5322f725ea92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:26.311479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e983d-88ed-4fcd-b7ec-5322f725ea92 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:26.311792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e983d-88ed-4fcd-b7ec-5322f725ea92 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:26.312047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e983d-88ed-4fcd-b7ec-5322f725ea92 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:26.312665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e983d-88ed-4fcd-b7ec-5322f725ea92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:26.314126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e983d-88ed-4fcd-b7ec-5322f725ea92 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:26.329691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.sql.core [None req-955e983d-88ed-4fcd-b7ec-5322f725ea92 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 21:01:26.329691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 21:01:26.329691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [parameters: {'user_id': '6f3681a3fca14bb8a5a745520b28d4ad', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 21:01:26.329691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81697) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 21:01:26.332180 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52f25e-6062-486c-854c-4fcf63b2108e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:26.333150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52f25e-6062-486c-854c-4fcf63b2108e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:26.333603 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52f25e-6062-486c-854c-4fcf63b2108e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:26.334095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52f25e-6062-486c-854c-4fcf63b2108e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-955e983d-88ed-4fcd-b7ec-5322f725ea92 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Sep 06 21:01:26.336868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 21:01:26.337765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application yield resource Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise self.value Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application session.commit() Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 21:01:26.339044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.flush() Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application compat.raise_( Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 21:01:26.340311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 21:01:26.341336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 21:01:26.342183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [parameters: {'user_id': '6f3681a3fca14bb8a5a745520b28d4ad', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:26.343043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 06 21:01:26.343875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 06 21:01:26.345198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 06 21:01:26.345198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 06 21:01:26.345198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 21:01:26.345198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 21:01:26.345198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 21:01:26.345198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:26.345198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 486/981] 10.222.0.79 () {60 vars in 1131 bytes} [Tue Sep 6 21:01:26 2022] POST /identity/v3/users => generated 160 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:26.345198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bc2f8-a8e6-4206-a833-9de606b012e1 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:26.345198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bc2f8-a8e6-4206-a833-9de606b012e1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:26.345198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bc2f8-a8e6-4206-a833-9de606b012e1 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:26.345752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 487/982] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:26.349468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bc2f8-a8e6-4206-a833-9de606b012e1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:26.354273 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52f25e-6062-486c-854c-4fcf63b2108e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:26.355992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52f25e-6062-486c-854c-4fcf63b2108e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5a52f25e-6062-486c-854c-4fcf63b2108e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:01:26.371572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:26.372722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:26.373796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:01:26.373796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:01:26.373796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:01:26.373796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:01:26.373796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:01:26.373796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:01:26.373796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:26.373796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:26.376239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 496/983] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:26.378265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330abc71-6466-414b-b274-9c1c77c5cddf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:26.378766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330abc71-6466-414b-b274-9c1c77c5cddf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:26.379421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330abc71-6466-414b-b274-9c1c77c5cddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:26.405907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-330abc71-6466-414b-b274-9c1c77c5cddf None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:26.407636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-330abc71-6466-414b-b274-9c1c77c5cddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:26.460024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:26.470788 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bc2f8-a8e6-4206-a833-9de606b012e1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:26.485571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 497/984] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:26.486715 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330abc71-6466-414b-b274-9c1c77c5cddf None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:26.516204 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a36171-703e-44b2-9fbc-504e868f4631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:26.516517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a36171-703e-44b2-9fbc-504e868f4631 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:26.516824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a36171-703e-44b2-9fbc-504e868f4631 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:26.517133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a36171-703e-44b2-9fbc-504e868f4631 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:26.525216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a36171-703e-44b2-9fbc-504e868f4631 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:26.526775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a36171-703e-44b2-9fbc-504e868f4631 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a2a36171-703e-44b2-9fbc-504e868f4631 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:01:26.539879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:26.540885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:26.541844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:26.541844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:26.541844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:01:26.541844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:01:26.541844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:01:26.541844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:01:26.541844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:01:26.541844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:01:26.541844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 21:01:26.541844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:26.545424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 488/985] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:01:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:26.546295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a36171-703e-44b2-9fbc-504e868f4631 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:26.610287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:26.621894 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330abc71-6466-414b-b274-9c1c77c5cddf None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:26.658159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2477c3b-bf38-444c-9f5f-cf54d8e5c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:26.658832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2477c3b-bf38-444c-9f5f-cf54d8e5c68f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:26.659108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2477c3b-bf38-444c-9f5f-cf54d8e5c68f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:26.659490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2477c3b-bf38-444c-9f5f-cf54d8e5c68f None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:26.660302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2477c3b-bf38-444c-9f5f-cf54d8e5c68f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:26.660611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2477c3b-bf38-444c-9f5f-cf54d8e5c68f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:26.662090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2477c3b-bf38-444c-9f5f-cf54d8e5c68f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:26.676043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 498/986] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:26.680297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a36171-703e-44b2-9fbc-504e868f4631 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:26.681654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2477c3b-bf38-444c-9f5f-cf54d8e5c68f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:26.727011 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f56a9e-243b-43ff-b518-3f82524f0ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:26.727674 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f56a9e-243b-43ff-b518-3f82524f0ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:26.727955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f56a9e-243b-43ff-b518-3f82524f0ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:26.728226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f56a9e-243b-43ff-b518-3f82524f0ed9 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:26.728968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f56a9e-243b-43ff-b518-3f82524f0ed9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:26.729268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f56a9e-243b-43ff-b518-3f82524f0ed9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:26.730831 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f56a9e-243b-43ff-b518-3f82524f0ed9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:26.734530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:26.774302 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 489/987] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Sep 6 21:01:26 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:26.805798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f56a9e-243b-43ff-b518-3f82524f0ed9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:26.815646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2477c3b-bf38-444c-9f5f-cf54d8e5c68f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:26.856131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49726d4-50b7-416d-998b-e8afce89a785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:26.856955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49726d4-50b7-416d-998b-e8afce89a785 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:26.857346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49726d4-50b7-416d-998b-e8afce89a785 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:26.857707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49726d4-50b7-416d-998b-e8afce89a785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:26.858552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49726d4-50b7-416d-998b-e8afce89a785 None admin] RBAC: Adding query filter params (role.id=6bc0365fc21845598826c2848897e72b, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=c09b4900d97b4b35b617af9a42baf9ee) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:26.858952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49726d4-50b7-416d-998b-e8afce89a785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:26.860523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49726d4-50b7-416d-998b-e8afce89a785 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:26.872701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 499/988] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:01:26 2022] GET /identity/v3/role_assignments?user.id=c09b4900d97b4b35b617af9a42baf9ee&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=6bc0365fc21845598826c2848897e72b => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:26.935814 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:26.940692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f56a9e-243b-43ff-b518-3f82524f0ed9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:26.990300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9736bfa-b311-4a7d-bb5e-8ffc5df54dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:26.990953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9736bfa-b311-4a7d-bb5e-8ffc5df54dd5 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:26.991256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9736bfa-b311-4a7d-bb5e-8ffc5df54dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:26.991544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9736bfa-b311-4a7d-bb5e-8ffc5df54dd5 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:26.992560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9736bfa-b311-4a7d-bb5e-8ffc5df54dd5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:26.995312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9736bfa-b311-4a7d-bb5e-8ffc5df54dd5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:27.015332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 490/989] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 21:01:26 2022] POST /identity/v3/services => generated 242 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:01:28.357576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefea36a-d6c6-4575-893b-3f9baa626cc1 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:28.358185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefea36a-d6c6-4575-893b-3f9baa626cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:28.358185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefea36a-d6c6-4575-893b-3f9baa626cc1 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:28.359759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 500/990] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:28.365438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54deea67-af35-4518-8de2-2f15fbcb9489 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:28.365948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54deea67-af35-4518-8de2-2f15fbcb9489 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:28.366027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54deea67-af35-4518-8de2-2f15fbcb9489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:28.384772 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-54deea67-af35-4518-8de2-2f15fbcb9489 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:28.386318 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-54deea67-af35-4518-8de2-2f15fbcb9489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:28.467727 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 491/991] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:28.729582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefea36a-d6c6-4575-893b-3f9baa626cc1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:28.842746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefea36a-d6c6-4575-893b-3f9baa626cc1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:28.845249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:28.878815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94eb4b-c77c-4242-88f0-7aa8a5e157c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:28.879456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94eb4b-c77c-4242-88f0-7aa8a5e157c3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:28.879733 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94eb4b-c77c-4242-88f0-7aa8a5e157c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:28.880003 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94eb4b-c77c-4242-88f0-7aa8a5e157c3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:28.880589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94eb4b-c77c-4242-88f0-7aa8a5e157c3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:28.882001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94eb4b-c77c-4242-88f0-7aa8a5e157c3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6f94eb4b-c77c-4242-88f0-7aa8a5e157c3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:28.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:28.893115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 21:01:28.894021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:28.896843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 501/992] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Sep 6 21:01:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:28.900156 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54deea67-af35-4518-8de2-2f15fbcb9489 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:28.991501 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:29.018672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54deea67-af35-4518-8de2-2f15fbcb9489 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:29.058257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0689e6d-0557-4956-961c-e8d7b7ebb68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:29.058882 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0689e6d-0557-4956-961c-e8d7b7ebb68c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:29.059158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0689e6d-0557-4956-961c-e8d7b7ebb68c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:29.059424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0689e6d-0557-4956-961c-e8d7b7ebb68c None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:29.060446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0689e6d-0557-4956-961c-e8d7b7ebb68c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:29.060776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0689e6d-0557-4956-961c-e8d7b7ebb68c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:29.062171 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0689e6d-0557-4956-961c-e8d7b7ebb68c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:29.067960 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 492/993] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Sep 6 21:01:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:29.074004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94eb4b-c77c-4242-88f0-7aa8a5e157c3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:29.125078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5aed4-4512-429b-973d-13987603b0b2 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:29.125399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5aed4-4512-429b-973d-13987603b0b2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:29.125621 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5aed4-4512-429b-973d-13987603b0b2 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:29.127104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 493/994] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:01:29.132014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb2c2b-f0b7-4f2e-821b-42423b716b64 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:29.132295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb2c2b-f0b7-4f2e-821b-42423b716b64 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:29.132568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb2c2b-f0b7-4f2e-821b-42423b716b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:29.151465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5ecb2c2b-f0b7-4f2e-821b-42423b716b64 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:29.155428 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5ecb2c2b-f0b7-4f2e-821b-42423b716b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:29.200953 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94eb4b-c77c-4242-88f0-7aa8a5e157c3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:29.241802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af813b-4355-42b2-8872-7c018c22552d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:29.242329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af813b-4355-42b2-8872-7c018c22552d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:29.242610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af813b-4355-42b2-8872-7c018c22552d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:29.242884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af813b-4355-42b2-8872-7c018c22552d None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:29.243628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af813b-4355-42b2-8872-7c018c22552d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:29.243933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af813b-4355-42b2-8872-7c018c22552d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:29.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af813b-4355-42b2-8872-7c018c22552d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:29.249778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 494/995] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:29.250962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 502/996] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Sep 6 21:01:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:29.251064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa58f49-dc64-4eed-9ed2-3e7b5422c121 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:29.251137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa58f49-dc64-4eed-9ed2-3e7b5422c121 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:29.251399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa58f49-dc64-4eed-9ed2-3e7b5422c121 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:29.252684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 495/997] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:29.258217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993f352-d0c7-4a76-84ba-2252045036e6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:29.258712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993f352-d0c7-4a76-84ba-2252045036e6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:29.258991 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993f352-d0c7-4a76-84ba-2252045036e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:29.259231 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa58f49-dc64-4eed-9ed2-3e7b5422c121 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:29.279220 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e993f352-d0c7-4a76-84ba-2252045036e6 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:29.280576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e993f352-d0c7-4a76-84ba-2252045036e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:29.328530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:29.369285 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:29.374743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 503/998] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:29.405936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa58f49-dc64-4eed-9ed2-3e7b5422c121 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:29.445412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61706c25-be12-48a4-9d47-4cf221defda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:29.446579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61706c25-be12-48a4-9d47-4cf221defda7 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:29.446579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61706c25-be12-48a4-9d47-4cf221defda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:29.446579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61706c25-be12-48a4-9d47-4cf221defda7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:29.447229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61706c25-be12-48a4-9d47-4cf221defda7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:29.448597 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61706c25-be12-48a4-9d47-4cf221defda7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:29.471587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 496/999] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:01:29 2022] POST /identity/v3/endpoints => generated 325 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:29.528033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993f352-d0c7-4a76-84ba-2252045036e6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:29.640975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61706c25-be12-48a4-9d47-4cf221defda7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:29.657906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993f352-d0c7-4a76-84ba-2252045036e6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:29.712550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b2618-9b65-466e-ad2d-e21b05812606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:29.713248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b2618-9b65-466e-ad2d-e21b05812606 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:29.713494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b2618-9b65-466e-ad2d-e21b05812606 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:29.713758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b2618-9b65-466e-ad2d-e21b05812606 None admin] PATH_INFO: `/v3/services/image` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:29.714339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b2618-9b65-466e-ad2d-e21b05812606 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:29.715719 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b2618-9b65-466e-ad2d-e21b05812606 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f32b2618-9b65-466e-ad2d-e21b05812606 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:29.725004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:29.726022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:29.727033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 504/1000] 10.222.0.79 () {56 vars in 1105 bytes} [Tue Sep 6 21:01:29 2022] GET /identity/v3/services/image => generated 86 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:29.732302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b2618-9b65-466e-ad2d-e21b05812606 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:29.764009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:29.784345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61706c25-be12-48a4-9d47-4cf221defda7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:29.822349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5605e-5bf3-4e26-abdf-dd5867b40729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:29.822997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5605e-5bf3-4e26-abdf-dd5867b40729 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:29.823308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5605e-5bf3-4e26-abdf-dd5867b40729 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:29.823679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5605e-5bf3-4e26-abdf-dd5867b40729 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:29.824586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5605e-5bf3-4e26-abdf-dd5867b40729 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:29.826162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5605e-5bf3-4e26-abdf-dd5867b40729 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7ed5605e-5bf3-4e26-abdf-dd5867b40729 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:29.836232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:29.837492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:29.842436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 497/1001] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:29.844688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5605e-5bf3-4e26-abdf-dd5867b40729 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:29.848422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:29.859015 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b2618-9b65-466e-ad2d-e21b05812606 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:29.897830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1101d29-3449-49b5-a7de-4dea6c768fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:29.898821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1101d29-3449-49b5-a7de-4dea6c768fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:29.899102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1101d29-3449-49b5-a7de-4dea6c768fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:29.899382 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1101d29-3449-49b5-a7de-4dea6c768fd0 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:29.900425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1101d29-3449-49b5-a7de-4dea6c768fd0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:29.900758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1101d29-3449-49b5-a7de-4dea6c768fd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:29.902696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1101d29-3449-49b5-a7de-4dea6c768fd0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:29.909484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 505/1002] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:29.915576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1101d29-3449-49b5-a7de-4dea6c768fd0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:29.959032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5605e-5bf3-4e26-abdf-dd5867b40729 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:29.996778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3dc93-0879-4917-a721-a496da3e9e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:29.997398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3dc93-0879-4917-a721-a496da3e9e02 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:29.997667 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3dc93-0879-4917-a721-a496da3e9e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:29.997932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3dc93-0879-4917-a721-a496da3e9e02 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:29.998561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3dc93-0879-4917-a721-a496da3e9e02 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:29.998857 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3dc93-0879-4917-a721-a496da3e9e02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:30.000232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3dc93-0879-4917-a721-a496da3e9e02 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:30.005797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 498/1003] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Sep 6 21:01:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:30.011487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3dc93-0879-4917-a721-a496da3e9e02 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:30.019889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:30.037453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1101d29-3449-49b5-a7de-4dea6c768fd0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:30.077218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0f2e8-f754-43ee-a32a-35643214ce19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:30.077822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0f2e8-f754-43ee-a32a-35643214ce19 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:30.078098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0f2e8-f754-43ee-a32a-35643214ce19 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:30.078360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0f2e8-f754-43ee-a32a-35643214ce19 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:30.078989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0f2e8-f754-43ee-a32a-35643214ce19 None admin] RBAC: Adding query filter params (type=image) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:30.079284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0f2e8-f754-43ee-a32a-35643214ce19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:30.080676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0f2e8-f754-43ee-a32a-35643214ce19 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:30.085630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:30.085787 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 506/1004] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:30.146410 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3dc93-0879-4917-a721-a496da3e9e02 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:30.195298 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca98efd-27b4-4da8-bc53-734590b02cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:30.196184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca98efd-27b4-4da8-bc53-734590b02cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:30.196636 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca98efd-27b4-4da8-bc53-734590b02cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:30.197157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca98efd-27b4-4da8-bc53-734590b02cb4 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:30.197955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca98efd-27b4-4da8-bc53-734590b02cb4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:30.198401 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca98efd-27b4-4da8-bc53-734590b02cb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:30.199961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca98efd-27b4-4da8-bc53-734590b02cb4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:30.206622 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 499/1005] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Sep 6 21:01:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:30.213164 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0f2e8-f754-43ee-a32a-35643214ce19 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:30.325566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0f2e8-f754-43ee-a32a-35643214ce19 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:30.363755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526810e-9491-438c-9a9e-ed4b99dfec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:30.364365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526810e-9491-438c-9a9e-ed4b99dfec0c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:30.364636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526810e-9491-438c-9a9e-ed4b99dfec0c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:30.364925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526810e-9491-438c-9a9e-ed4b99dfec0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:30.365625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526810e-9491-438c-9a9e-ed4b99dfec0c None admin] RBAC: Adding query filter params (interface=public, service_id=b6ce9a409d484d5d9888434e48d2cfef, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:30.365923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526810e-9491-438c-9a9e-ed4b99dfec0c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:30.367530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526810e-9491-438c-9a9e-ed4b99dfec0c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:30.373413 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 507/1006] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Sep 6 21:01:30 2022] GET /identity/v3/endpoints?service_id=b6ce9a409d484d5d9888434e48d2cfef&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:30.379244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca98efd-27b4-4da8-bc53-734590b02cb4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:30.486826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca98efd-27b4-4da8-bc53-734590b02cb4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:30.519679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:30.527067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59063ff9-1c3b-45dc-b5e0-cd2e2acb3708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:30.527711 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59063ff9-1c3b-45dc-b5e0-cd2e2acb3708 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:30.527985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59063ff9-1c3b-45dc-b5e0-cd2e2acb3708 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:30.528252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59063ff9-1c3b-45dc-b5e0-cd2e2acb3708 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:30.529134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59063ff9-1c3b-45dc-b5e0-cd2e2acb3708 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:30.530274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59063ff9-1c3b-45dc-b5e0-cd2e2acb3708 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:30.537550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 500/1007] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Sep 6 21:01:30 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:01:31.662029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a8a7e-6337-42ad-9243-b67e460d2df4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:31.662574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a8a7e-6337-42ad-9243-b67e460d2df4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:31.662574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a8a7e-6337-42ad-9243-b67e460d2df4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:31.664107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 508/1008] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:31.669601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc5b9b-cbfd-4fac-b88a-f8c24fc38143 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:31.670047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc5b9b-cbfd-4fac-b88a-f8c24fc38143 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:31.670148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc5b9b-cbfd-4fac-b88a-f8c24fc38143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:31.689454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f9cc5b9b-cbfd-4fac-b88a-f8c24fc38143 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:31.690738 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f9cc5b9b-cbfd-4fac-b88a-f8c24fc38143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:31.773016 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 501/1009] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:31 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:32.035001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a8a7e-6337-42ad-9243-b67e460d2df4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:32.051973 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:32.150809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a8a7e-6337-42ad-9243-b67e460d2df4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:32.191242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39008b40-e6b4-4e37-8e13-f7524a808ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:32.192176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39008b40-e6b4-4e37-8e13-f7524a808ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:32.192602 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39008b40-e6b4-4e37-8e13-f7524a808ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:32.193133 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39008b40-e6b4-4e37-8e13-f7524a808ee8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:32.194554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39008b40-e6b4-4e37-8e13-f7524a808ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:32.196140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39008b40-e6b4-4e37-8e13-f7524a808ee8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:32.198241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 509/1010] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:32.204062 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc5b9b-cbfd-4fac-b88a-f8c24fc38143 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:32.258114 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfcd9b-6f3a-457b-b4b7-46d8d5f14bd5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:32.258409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfcd9b-6f3a-457b-b4b7-46d8d5f14bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:32.258669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfcd9b-6f3a-457b-b4b7-46d8d5f14bd5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:32.260231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 510/1011] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:32.265334 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7945c-0977-43ac-ab33-e15392187284 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:32.265619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7945c-0977-43ac-ab33-e15392187284 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:32.265891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7945c-0977-43ac-ab33-e15392187284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:32.284807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-47c7945c-0977-43ac-ab33-e15392187284 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:32.286027 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-47c7945c-0977-43ac-ab33-e15392187284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:32.323638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc5b9b-cbfd-4fac-b88a-f8c24fc38143 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:32.351355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:32.365138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b93fce-2b7c-4f31-83d7-372cf3800084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:32.365846 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b93fce-2b7c-4f31-83d7-372cf3800084 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:32.366122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b93fce-2b7c-4f31-83d7-372cf3800084 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:32.366385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b93fce-2b7c-4f31-83d7-372cf3800084 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:32.367077 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b93fce-2b7c-4f31-83d7-372cf3800084 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:32.372702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 511/1012] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:32 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:32.372865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b93fce-2b7c-4f31-83d7-372cf3800084 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:32.402056 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 502/1013] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 21:01:32 2022] POST /identity/v3/projects => generated 311 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:32.638148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7945c-0977-43ac-ab33-e15392187284 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:32.653686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:32.732796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f778c22-edb6-42d9-b4e1-837698e36273 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:32.733462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f778c22-edb6-42d9-b4e1-837698e36273 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:32.733722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f778c22-edb6-42d9-b4e1-837698e36273 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:32.738454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 503/1014] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:01:32.744191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035450b-0754-4621-9674-fe18fcacd0a4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:32.744472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035450b-0754-4621-9674-fe18fcacd0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:32.744783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035450b-0754-4621-9674-fe18fcacd0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:32.762694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7945c-0977-43ac-ab33-e15392187284 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:32.763541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a035450b-0754-4621-9674-fe18fcacd0a4 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:32.764951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a035450b-0754-4621-9674-fe18fcacd0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:32.814879 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac23844-9aa1-481e-a641-7e857346e750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:32.815779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac23844-9aa1-481e-a641-7e857346e750 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:32.816241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac23844-9aa1-481e-a641-7e857346e750 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:32.816511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac23844-9aa1-481e-a641-7e857346e750 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:32.817210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac23844-9aa1-481e-a641-7e857346e750 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:32.818669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac23844-9aa1-481e-a641-7e857346e750 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:32.836936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 512/1015] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 21:01:32 2022] POST /identity/v3/services => generated 242 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:01:32.870289 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 504/1016] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:32 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:33.144896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac23844-9aa1-481e-a641-7e857346e750 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:33.227115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:33.263096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac23844-9aa1-481e-a641-7e857346e750 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:33.297903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847164d-fddb-427d-b460-da07d8fef257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:33.298554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847164d-fddb-427d-b460-da07d8fef257 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:33.298835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847164d-fddb-427d-b460-da07d8fef257 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:33.299108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847164d-fddb-427d-b460-da07d8fef257 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:33.299704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847164d-fddb-427d-b460-da07d8fef257 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:33.301148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847164d-fddb-427d-b460-da07d8fef257 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5847164d-fddb-427d-b460-da07d8fef257 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:33.312092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:33.313274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 21:01:33.314109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:33.316725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 513/1017] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:33.319377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035450b-0754-4621-9674-fe18fcacd0a4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:33.452575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035450b-0754-4621-9674-fe18fcacd0a4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:33.454133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:33.497010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c615d-710c-480a-9bbb-ef4150b59733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:33.497908 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c615d-710c-480a-9bbb-ef4150b59733 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:33.498319 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c615d-710c-480a-9bbb-ef4150b59733 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:33.498717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c615d-710c-480a-9bbb-ef4150b59733 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:33.499718 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c615d-710c-480a-9bbb-ef4150b59733 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:33.500166 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c615d-710c-480a-9bbb-ef4150b59733 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:33.502364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c615d-710c-480a-9bbb-ef4150b59733 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:33.514964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 505/1018] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Sep 6 21:01:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:33.516309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847164d-fddb-427d-b460-da07d8fef257 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:33.636648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847164d-fddb-427d-b460-da07d8fef257 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:33.687003 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a04eaa-5262-4002-a596-00e5e7cf80d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:33.687627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a04eaa-5262-4002-a596-00e5e7cf80d4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:33.687901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a04eaa-5262-4002-a596-00e5e7cf80d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:33.688167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a04eaa-5262-4002-a596-00e5e7cf80d4 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:33.688828 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a04eaa-5262-4002-a596-00e5e7cf80d4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:33.689128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a04eaa-5262-4002-a596-00e5e7cf80d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:33.690510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a04eaa-5262-4002-a596-00e5e7cf80d4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:33.692873 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:33.701424 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 514/1019] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Sep 6 21:01:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:33.703354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c615d-710c-480a-9bbb-ef4150b59733 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:33.826003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c615d-710c-480a-9bbb-ef4150b59733 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:33.867249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04497fb-57b3-47d2-a5d8-8c64def4a81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:33.867861 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04497fb-57b3-47d2-a5d8-8c64def4a81c None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:33.868135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04497fb-57b3-47d2-a5d8-8c64def4a81c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:33.868405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04497fb-57b3-47d2-a5d8-8c64def4a81c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:33.869206 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04497fb-57b3-47d2-a5d8-8c64def4a81c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:33.870588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04497fb-57b3-47d2-a5d8-8c64def4a81c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:33.900715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 506/1020] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:01:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:34.700116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7c299-4eb3-4c8d-91cf-dd17054a2b70 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:34.700943 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7c299-4eb3-4c8d-91cf-dd17054a2b70 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:34.700943 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7c299-4eb3-4c8d-91cf-dd17054a2b70 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:34.702134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 515/1021] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:34.707920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45f2fc-c2e7-4758-a17f-6ffdc5d3476b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:34.708217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45f2fc-c2e7-4758-a17f-6ffdc5d3476b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:34.708460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45f2fc-c2e7-4758-a17f-6ffdc5d3476b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:34.728312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fd45f2fc-c2e7-4758-a17f-6ffdc5d3476b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:34.729549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fd45f2fc-c2e7-4758-a17f-6ffdc5d3476b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:34.749327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:34.820359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 507/1022] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:35.053996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41314431-c491-4233-b86a-4165c8f4949f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:35.054427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41314431-c491-4233-b86a-4165c8f4949f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:35.054532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41314431-c491-4233-b86a-4165c8f4949f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:35.056080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 516/1023] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:35.062048 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88aa7fd-6ff0-4a12-8eda-b0ead3534199 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:35.062545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88aa7fd-6ff0-4a12-8eda-b0ead3534199 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:35.062626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88aa7fd-6ff0-4a12-8eda-b0ead3534199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:35.082206 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f88aa7fd-6ff0-4a12-8eda-b0ead3534199 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:35.083503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f88aa7fd-6ff0-4a12-8eda-b0ead3534199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:35.084210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41314431-c491-4233-b86a-4165c8f4949f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:35.162760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 508/1024] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:35.195776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41314431-c491-4233-b86a-4165c8f4949f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:35.234685 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83b2f4-d7ae-4958-9184-cf7e32ba811d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:35.235333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83b2f4-d7ae-4958-9184-cf7e32ba811d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:35.235609 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83b2f4-d7ae-4958-9184-cf7e32ba811d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:35.235849 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83b2f4-d7ae-4958-9184-cf7e32ba811d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:35.237120 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83b2f4-d7ae-4958-9184-cf7e32ba811d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:35.238623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83b2f4-d7ae-4958-9184-cf7e32ba811d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:35.240494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 517/1025] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:35.246268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88aa7fd-6ff0-4a12-8eda-b0ead3534199 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:35.328787 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:35.353416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88aa7fd-6ff0-4a12-8eda-b0ead3534199 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:35.390147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b5344-0d9c-4f47-85fe-e7dcb562f2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:35.390753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b5344-0d9c-4f47-85fe-e7dcb562f2b4 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:35.391025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b5344-0d9c-4f47-85fe-e7dcb562f2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:35.391291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b5344-0d9c-4f47-85fe-e7dcb562f2b4 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:35.391969 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b5344-0d9c-4f47-85fe-e7dcb562f2b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:35.393431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b5344-0d9c-4f47-85fe-e7dcb562f2b4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:35.420706 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 509/1026] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 21:01:35 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:35.422465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83b2f4-d7ae-4958-9184-cf7e32ba811d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:35.500878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:35.539272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83b2f4-d7ae-4958-9184-cf7e32ba811d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:35.579572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3105d6f-96eb-4274-ac98-56aa3b727dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:35.580139 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105d6f-96eb-4274-ac98-56aa3b727dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:35.580440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105d6f-96eb-4274-ac98-56aa3b727dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:35.584140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105d6f-96eb-4274-ac98-56aa3b727dc8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:35.584140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3105d6f-96eb-4274-ac98-56aa3b727dc8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:35.584140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3105d6f-96eb-4274-ac98-56aa3b727dc8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d3105d6f-96eb-4274-ac98-56aa3b727dc8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:35.597031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:35.598143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 21:01:35.599004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:35.603365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 518/1027] 10.222.0.79 () {56 vars in 1105 bytes} [Tue Sep 6 21:01:35 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:35.604270 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b5344-0d9c-4f47-85fe-e7dcb562f2b4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:35.715471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b5344-0d9c-4f47-85fe-e7dcb562f2b4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:35.751730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b507c-6a5b-436f-864d-ee336b12ba9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:35.752356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b507c-6a5b-436f-864d-ee336b12ba9f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:35.752630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b507c-6a5b-436f-864d-ee336b12ba9f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:35.752924 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b507c-6a5b-436f-864d-ee336b12ba9f None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:35.753567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b507c-6a5b-436f-864d-ee336b12ba9f None admin] RBAC: Adding query filter params (name=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:35.753869 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b507c-6a5b-436f-864d-ee336b12ba9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:35.755254 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b507c-6a5b-436f-864d-ee336b12ba9f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:35.756610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:35.767757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 510/1028] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:35.769135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3105d6f-96eb-4274-ac98-56aa3b727dc8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:35.874907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3105d6f-96eb-4274-ac98-56aa3b727dc8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:35.911422 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:35.919474 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56de9b-5f3b-4199-aafd-8a4ffee851b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:35.920500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56de9b-5f3b-4199-aafd-8a4ffee851b7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:35.921005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56de9b-5f3b-4199-aafd-8a4ffee851b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:35.921474 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56de9b-5f3b-4199-aafd-8a4ffee851b7 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:35.922623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56de9b-5f3b-4199-aafd-8a4ffee851b7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:35.923135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56de9b-5f3b-4199-aafd-8a4ffee851b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:35.925710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56de9b-5f3b-4199-aafd-8a4ffee851b7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:35.939886 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 519/1029] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:35.941066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b507c-6a5b-436f-864d-ee336b12ba9f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:36.049058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b507c-6a5b-436f-864d-ee336b12ba9f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:36.087983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c4f2f-4d4d-4036-b2b3-c1386301c2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:36.088796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c4f2f-4d4d-4036-b2b3-c1386301c2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:36.089055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c4f2f-4d4d-4036-b2b3-c1386301c2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:36.089226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c4f2f-4d4d-4036-b2b3-c1386301c2e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:36.089980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c4f2f-4d4d-4036-b2b3-c1386301c2e7 None admin] RBAC: Adding query filter params (interface=public, service_id=8ae90fbe7eea47b8ae4f74dcc08298ff, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:36.090300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c4f2f-4d4d-4036-b2b3-c1386301c2e7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:36.091693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c4f2f-4d4d-4036-b2b3-c1386301c2e7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:36.097569 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 511/1030] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Sep 6 21:01:35 2022] GET /identity/v3/endpoints?service_id=8ae90fbe7eea47b8ae4f74dcc08298ff&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:36.103275 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56de9b-5f3b-4199-aafd-8a4ffee851b7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:36.176731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed1cfd-f5bf-400d-afbc-f87d5083b0e5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:36.177231 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed1cfd-f5bf-400d-afbc-f87d5083b0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:36.177564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed1cfd-f5bf-400d-afbc-f87d5083b0e5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:36.179055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 512/1031] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:36.186627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed19492-c117-47d0-9a61-e613debd3bb4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:36.187116 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed19492-c117-47d0-9a61-e613debd3bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:36.187448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed19492-c117-47d0-9a61-e613debd3bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:36.212242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0ed19492-c117-47d0-9a61-e613debd3bb4 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:36.214078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0ed19492-c117-47d0-9a61-e613debd3bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:36.218929 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56de9b-5f3b-4199-aafd-8a4ffee851b7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:36.262703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef59dcd-f6bd-4dd5-a9b4-83960b2b267d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:36.263330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef59dcd-f6bd-4dd5-a9b4-83960b2b267d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:36.263607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef59dcd-f6bd-4dd5-a9b4-83960b2b267d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:36.263877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef59dcd-f6bd-4dd5-a9b4-83960b2b267d None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:36.264491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef59dcd-f6bd-4dd5-a9b4-83960b2b267d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:36.266137 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef59dcd-f6bd-4dd5-a9b4-83960b2b267d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:36.273530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 520/1032] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Sep 6 21:01:36 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:01:36.303623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:36.328691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 513/1033] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:36.599006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef59dcd-f6bd-4dd5-a9b4-83960b2b267d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:36.665496 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:36.713009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef59dcd-f6bd-4dd5-a9b4-83960b2b267d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:36.750922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866b55d-e499-4108-a8a2-31b3409f6e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:36.751576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866b55d-e499-4108-a8a2-31b3409f6e94 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:36.751853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866b55d-e499-4108-a8a2-31b3409f6e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:36.752121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866b55d-e499-4108-a8a2-31b3409f6e94 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:36.763059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866b55d-e499-4108-a8a2-31b3409f6e94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4866b55d-e499-4108-a8a2-31b3409f6e94 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:36.768514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:36.770124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:36.770974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:36.770974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:36.770974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:36.770974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:36.770974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:36.770974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:36.770974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:36.770974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:36.770974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:36.770974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:36.770974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 21:01:36.770974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:36.772748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 521/1034] 10.222.0.79 () {56 vars in 1120 bytes} [Tue Sep 6 21:01:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:36.776956 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed19492-c117-47d0-9a61-e613debd3bb4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:36.887045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed19492-c117-47d0-9a61-e613debd3bb4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:36.929586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1ba23-176b-4c88-8e0f-fee207bf66fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:36.930199 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1ba23-176b-4c88-8e0f-fee207bf66fe None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:36.930480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1ba23-176b-4c88-8e0f-fee207bf66fe None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:36.930752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1ba23-176b-4c88-8e0f-fee207bf66fe None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:36.931584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1ba23-176b-4c88-8e0f-fee207bf66fe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:36.931890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1ba23-176b-4c88-8e0f-fee207bf66fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:36.933311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1ba23-176b-4c88-8e0f-fee207bf66fe None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:36.942635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 514/1035] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Sep 6 21:01:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:36.948687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866b55d-e499-4108-a8a2-31b3409f6e94 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:37.052955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866b55d-e499-4108-a8a2-31b3409f6e94 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:37.074240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:37.089872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d722e-7954-4b0e-a23e-c06979e2c5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:37.090522 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d722e-7954-4b0e-a23e-c06979e2c5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:37.090797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d722e-7954-4b0e-a23e-c06979e2c5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:37.091067 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d722e-7954-4b0e-a23e-c06979e2c5d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:37.103554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d722e-7954-4b0e-a23e-c06979e2c5d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:37.105065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d722e-7954-4b0e-a23e-c06979e2c5d2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-048d722e-7954-4b0e-a23e-c06979e2c5d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:37.120085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:37.121464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:37.122727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:37.122727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:37.122727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:37.124750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 522/1036] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:37.127687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1ba23-176b-4c88-8e0f-fee207bf66fe None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:37.189718 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:37.239469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1ba23-176b-4c88-8e0f-fee207bf66fe None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:37.278829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec210c9-bc62-4097-bd95-b84e16f04ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:37.279453 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec210c9-bc62-4097-bd95-b84e16f04ecb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:37.279728 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec210c9-bc62-4097-bd95-b84e16f04ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:37.280000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec210c9-bc62-4097-bd95-b84e16f04ecb None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:37.280650 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec210c9-bc62-4097-bd95-b84e16f04ecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:37.280978 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec210c9-bc62-4097-bd95-b84e16f04ecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:37.282369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec210c9-bc62-4097-bd95-b84e16f04ecb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:37.295855 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 515/1037] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:37.301538 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d722e-7954-4b0e-a23e-c06979e2c5d2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:37.409354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d722e-7954-4b0e-a23e-c06979e2c5d2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:37.444038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733b429-f520-4e23-83dd-44674014baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:37.444593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733b429-f520-4e23-83dd-44674014baf5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:37.444900 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733b429-f520-4e23-83dd-44674014baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:37.445167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733b429-f520-4e23-83dd-44674014baf5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:37.453730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733b429-f520-4e23-83dd-44674014baf5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:37.455185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733b429-f520-4e23-83dd-44674014baf5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2733b429-f520-4e23-83dd-44674014baf5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:01:37.467106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:37.468103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:37.469472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:37.469472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:37.469472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:01:37.469472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:01:37.469472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:01:37.469472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:01:37.469472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:01:37.469472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:01:37.469472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:01:37.469472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:37.472748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 523/1038] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Sep 6 21:01:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:37.474279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec210c9-bc62-4097-bd95-b84e16f04ecb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:37.597100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec210c9-bc62-4097-bd95-b84e16f04ecb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:37.615748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:37.638180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1268588-bdaa-458c-8ae2-d3a7d6e532d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:37.638793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1268588-bdaa-458c-8ae2-d3a7d6e532d2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:37.639069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1268588-bdaa-458c-8ae2-d3a7d6e532d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:37.639336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1268588-bdaa-458c-8ae2-d3a7d6e532d2 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:37.640064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1268588-bdaa-458c-8ae2-d3a7d6e532d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:37.640366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1268588-bdaa-458c-8ae2-d3a7d6e532d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:37.641837 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1268588-bdaa-458c-8ae2-d3a7d6e532d2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:37.664848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 516/1039] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Sep 6 21:01:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:37.666507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733b429-f520-4e23-83dd-44674014baf5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:37.700710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:37.710334 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244664d-f57a-4aae-a824-7aac4899940f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:37.710630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244664d-f57a-4aae-a824-7aac4899940f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:37.710895 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244664d-f57a-4aae-a824-7aac4899940f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:37.712533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 517/1040] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:01:37.717693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e2403-3295-4dd1-aad1-f09b91c18538 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:37.717984 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e2403-3295-4dd1-aad1-f09b91c18538 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:37.718256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e2403-3295-4dd1-aad1-f09b91c18538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:37.737851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5d6e2403-3295-4dd1-aad1-f09b91c18538 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:37.739088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5d6e2403-3295-4dd1-aad1-f09b91c18538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:37.778448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733b429-f520-4e23-83dd-44674014baf5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:37.813279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 518/1041] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:37.816276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d8c9f-e1d2-47f3-ab9a-3d255deab523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:37.816988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d8c9f-e1d2-47f3-ab9a-3d255deab523 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:37.817269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d8c9f-e1d2-47f3-ab9a-3d255deab523 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:37.817540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d8c9f-e1d2-47f3-ab9a-3d255deab523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:37.895177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d8c9f-e1d2-47f3-ab9a-3d255deab523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:37.896935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d8c9f-e1d2-47f3-ab9a-3d255deab523 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:37.935790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:38.019269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 524/1042] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:01:37 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:01:38.025291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e2403-3295-4dd1-aad1-f09b91c18538 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:38.076038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d8c9f-e1d2-47f3-ab9a-3d255deab523 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:38.132427 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e2403-3295-4dd1-aad1-f09b91c18538 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:38.134199 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:38.171182 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf2094-8785-4ea3-9051-dcd93df8698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:38.171977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf2094-8785-4ea3-9051-dcd93df8698b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:38.172262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf2094-8785-4ea3-9051-dcd93df8698b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:38.172881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf2094-8785-4ea3-9051-dcd93df8698b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:38.181123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d8c9f-e1d2-47f3-ab9a-3d255deab523 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:38.183584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf2094-8785-4ea3-9051-dcd93df8698b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:38.185478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf2094-8785-4ea3-9051-dcd93df8698b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ceaf2094-8785-4ea3-9051-dcd93df8698b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:38.201363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:38.202387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:38.203565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:38.203565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:38.203565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:38.203565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 519/1043] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:38.208276 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf2094-8785-4ea3-9051-dcd93df8698b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:38.218741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:38.224456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2f77f-d69b-44a9-acf4-92087ea19bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:38.225133 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2f77f-d69b-44a9-acf4-92087ea19bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:38.225404 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2f77f-d69b-44a9-acf4-92087ea19bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:38.225671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2f77f-d69b-44a9-acf4-92087ea19bf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:38.239416 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2f77f-d69b-44a9-acf4-92087ea19bf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-04c2f77f-d69b-44a9-acf4-92087ea19bf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:38.244354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:38.245348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:38.246359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:38.246359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:38.246359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:38.246359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:38.246359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:38.246359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:38.246359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:38.246359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:38.246359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:38.246359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:38.246359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:01:38.246359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:38.246359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 525/1044] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:01:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:38.251918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2f77f-d69b-44a9-acf4-92087ea19bf3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:38.331595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf2094-8785-4ea3-9051-dcd93df8698b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:38.364867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2f77f-d69b-44a9-acf4-92087ea19bf3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:38.371128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1002c4-6b06-4aaa-bda7-1e9e560a2d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:38.371955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1002c4-6b06-4aaa-bda7-1e9e560a2d74 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:38.372940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1002c4-6b06-4aaa-bda7-1e9e560a2d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:38.372940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1002c4-6b06-4aaa-bda7-1e9e560a2d74 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:38.373291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1002c4-6b06-4aaa-bda7-1e9e560a2d74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:38.373463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1002c4-6b06-4aaa-bda7-1e9e560a2d74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:38.375033 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1002c4-6b06-4aaa-bda7-1e9e560a2d74 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:38.389333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 520/1045] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:38.395256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1002c4-6b06-4aaa-bda7-1e9e560a2d74 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:38.410171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b0b07-77c5-44cd-bb0e-c9f4fbb47105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:38.410843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b0b07-77c5-44cd-bb0e-c9f4fbb47105 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:38.411121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b0b07-77c5-44cd-bb0e-c9f4fbb47105 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:38.411444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b0b07-77c5-44cd-bb0e-c9f4fbb47105 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:38.412122 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b0b07-77c5-44cd-bb0e-c9f4fbb47105 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:38.412417 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b0b07-77c5-44cd-bb0e-c9f4fbb47105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:38.413516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:38.413868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b0b07-77c5-44cd-bb0e-c9f4fbb47105 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:38.423141 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 526/1046] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:01:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:38.429239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b0b07-77c5-44cd-bb0e-c9f4fbb47105 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:38.488253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:38.508554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1002c4-6b06-4aaa-bda7-1e9e560a2d74 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:38.532780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b0b07-77c5-44cd-bb0e-c9f4fbb47105 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:38.546227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b24c0b-22ae-4cf1-9bc2-58b6ad98003b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:38.546847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b24c0b-22ae-4cf1-9bc2-58b6ad98003b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:38.547115 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b24c0b-22ae-4cf1-9bc2-58b6ad98003b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:38.547376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b24c0b-22ae-4cf1-9bc2-58b6ad98003b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:38.554181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b24c0b-22ae-4cf1-9bc2-58b6ad98003b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:38.556026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b24c0b-22ae-4cf1-9bc2-58b6ad98003b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:38.566410 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e00ad-ad84-4ed6-a9b1-7b5a0448a159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:38.567044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e00ad-ad84-4ed6-a9b1-7b5a0448a159 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:38.567320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e00ad-ad84-4ed6-a9b1-7b5a0448a159 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:38.567587 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e00ad-ad84-4ed6-a9b1-7b5a0448a159 None admin] PATH_INFO: `/v3/users/3ac49e37b337489fa1ea68b29f2c548c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-90b24c0b-22ae-4cf1-9bc2-58b6ad98003b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:01:38.567756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:38.568598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:38.569743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:01:38.569743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:01:38.569743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:01:38.569743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:01:38.569743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:01:38.569743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:01:38.569743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:38.569743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:38.569743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 521/1047] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:38.573993 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408fc17-18db-4470-9757-bcaeaa599322 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:38.574346 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408fc17-18db-4470-9757-bcaeaa599322 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:38.574612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408fc17-18db-4470-9757-bcaeaa599322 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:38.575937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 522/1048] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:38.576619 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408fc17-18db-4470-9757-bcaeaa599322 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:38.583376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e00ad-ad84-4ed6-a9b1-7b5a0448a159 None admin] RBAC: Authorizing `identity:get_user(user_id=3ac49e37b337489fa1ea68b29f2c548c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:38.584807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e00ad-ad84-4ed6-a9b1-7b5a0448a159 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:38.586520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 527/1049] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:01:38 2022] GET /identity/v3/users/3ac49e37b337489fa1ea68b29f2c548c => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:38.587748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd837-a5d8-47ed-902c-4af6749d7c31 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:38.587955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd837-a5d8-47ed-902c-4af6749d7c31 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:38.588239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd837-a5d8-47ed-902c-4af6749d7c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:38.606002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9e2cd837-a5d8-47ed-902c-4af6749d7c31 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:38.607193 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9e2cd837-a5d8-47ed-902c-4af6749d7c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:38.668750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:38.680637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 528/1050] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:38.681513 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cd837-a5d8-47ed-902c-4af6749d7c31 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:38.700820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408fc17-18db-4470-9757-bcaeaa599322 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:38.734548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:38.741925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eab7ff-47cc-415e-ba23-54db0ab444b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:38.742537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eab7ff-47cc-415e-ba23-54db0ab444b5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:38.742815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eab7ff-47cc-415e-ba23-54db0ab444b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:38.743090 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eab7ff-47cc-415e-ba23-54db0ab444b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:38.743777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eab7ff-47cc-415e-ba23-54db0ab444b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:38.744095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eab7ff-47cc-415e-ba23-54db0ab444b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:38.745580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eab7ff-47cc-415e-ba23-54db0ab444b5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:38.757875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 523/1051] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:38.764315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eab7ff-47cc-415e-ba23-54db0ab444b5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:38.805982 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cd837-a5d8-47ed-902c-4af6749d7c31 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:38.844305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a22247-70ef-4085-beb4-3b241bfaaac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:38.849383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a22247-70ef-4085-beb4-3b241bfaaac7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:38.849383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a22247-70ef-4085-beb4-3b241bfaaac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:38.849383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a22247-70ef-4085-beb4-3b241bfaaac7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:38.879009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eab7ff-47cc-415e-ba23-54db0ab444b5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:38.914710 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc024c4-4497-4511-a2ef-f1bcea7b6677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:38.915324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc024c4-4497-4511-a2ef-f1bcea7b6677 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:38.915613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc024c4-4497-4511-a2ef-f1bcea7b6677 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:38.915859 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc024c4-4497-4511-a2ef-f1bcea7b6677 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:38.916631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc024c4-4497-4511-a2ef-f1bcea7b6677 None admin] RBAC: Adding query filter params (role.id=6bc0365fc21845598826c2848897e72b, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=12d51454d20c46d7a5e6d963a2f987cc) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:38.916987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc024c4-4497-4511-a2ef-f1bcea7b6677 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:38.918545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc024c4-4497-4511-a2ef-f1bcea7b6677 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:38.920718 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:38.937036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 524/1052] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:01:38 2022] GET /identity/v3/role_assignments?user.id=12d51454d20c46d7a5e6d963a2f987cc&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=6bc0365fc21845598826c2848897e72b => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:38.942101 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a22247-70ef-4085-beb4-3b241bfaaac7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:38.943616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a22247-70ef-4085-beb4-3b241bfaaac7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:38.951523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc024c4-4497-4511-a2ef-f1bcea7b6677 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:38.993960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:39.072240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc024c4-4497-4511-a2ef-f1bcea7b6677 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:39.079607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 529/1053] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:01:38 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:01:39.082475 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a22247-70ef-4085-beb4-3b241bfaaac7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:39.116113 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e3259-fbbd-4b79-a5d6-7de9b70b92ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:39.117039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e3259-fbbd-4b79-a5d6-7de9b70b92ee None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:39.117434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e3259-fbbd-4b79-a5d6-7de9b70b92ee None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:39.117582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e3259-fbbd-4b79-a5d6-7de9b70b92ee None admin] PATH_INFO: `/v3/services/image` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:39.119080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e3259-fbbd-4b79-a5d6-7de9b70b92ee None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:39.120517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e3259-fbbd-4b79-a5d6-7de9b70b92ee None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c37e3259-fbbd-4b79-a5d6-7de9b70b92ee None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:39.131421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:39.133750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 21:01:39.134672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:39.136123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 525/1054] 10.222.0.79 () {56 vars in 1105 bytes} [Tue Sep 6 21:01:38 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:39.139033 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e3259-fbbd-4b79-a5d6-7de9b70b92ee None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:39.200579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a22247-70ef-4085-beb4-3b241bfaaac7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:39.209943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:39.239818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f823f3-98df-402b-bf8f-36c46c81a6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:39.240457 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f823f3-98df-402b-bf8f-36c46c81a6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:39.240758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f823f3-98df-402b-bf8f-36c46c81a6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:39.241025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f823f3-98df-402b-bf8f-36c46c81a6e5 None admin] PATH_INFO: `/v3/projects/29972165e963424e832409079cb92fb2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:39.242223 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f823f3-98df-402b-bf8f-36c46c81a6e5 None admin] RBAC: Authorizing `identity:get_project(project_id=29972165e963424e832409079cb92fb2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:39.243847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f823f3-98df-402b-bf8f-36c46c81a6e5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:39.245791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 530/1055] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:01:39 2022] GET /identity/v3/projects/29972165e963424e832409079cb92fb2 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:39.251644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f823f3-98df-402b-bf8f-36c46c81a6e5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:39.256168 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e3259-fbbd-4b79-a5d6-7de9b70b92ee None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:39.265237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:39.294948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ee39a-278a-4864-a1e5-1320610b2a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:39.295564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ee39a-278a-4864-a1e5-1320610b2a2b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:39.295836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ee39a-278a-4864-a1e5-1320610b2a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:39.296101 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ee39a-278a-4864-a1e5-1320610b2a2b None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:39.296767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ee39a-278a-4864-a1e5-1320610b2a2b None admin] RBAC: Adding query filter params (name=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:39.297065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ee39a-278a-4864-a1e5-1320610b2a2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:39.298444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ee39a-278a-4864-a1e5-1320610b2a2b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:39.304171 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 526/1056] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:39.310418 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ee39a-278a-4864-a1e5-1320610b2a2b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:39.367636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f823f3-98df-402b-bf8f-36c46c81a6e5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:39.407278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedd115-9738-45ec-9177-ba67e87b89b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:39.407982 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedd115-9738-45ec-9177-ba67e87b89b7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:39.408297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedd115-9738-45ec-9177-ba67e87b89b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:39.408518 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedd115-9738-45ec-9177-ba67e87b89b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:39.409287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beedd115-9738-45ec-9177-ba67e87b89b7 None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=29972165e963424e832409079cb92fb2, user.id=3ac49e37b337489fa1ea68b29f2c548c) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:39.409693 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beedd115-9738-45ec-9177-ba67e87b89b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:39.411220 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beedd115-9738-45ec-9177-ba67e87b89b7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:39.417348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 531/1057] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:01:39 2022] GET /identity/v3/role_assignments?user.id=3ac49e37b337489fa1ea68b29f2c548c&scope.project.id=29972165e963424e832409079cb92fb2&role.id=3b30504991194002b41f8587d0851c10 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:39.427027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ee39a-278a-4864-a1e5-1320610b2a2b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:39.458535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:39.469931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885d005-2ffb-4804-b55a-8637255befec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:39.470569 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885d005-2ffb-4804-b55a-8637255befec None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:39.470849 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885d005-2ffb-4804-b55a-8637255befec None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:39.471122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885d005-2ffb-4804-b55a-8637255befec None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:39.471777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885d005-2ffb-4804-b55a-8637255befec None admin] RBAC: Adding query filter params (type=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:39.472079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885d005-2ffb-4804-b55a-8637255befec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:39.473621 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885d005-2ffb-4804-b55a-8637255befec None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:39.485110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 527/1058] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:39.486628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedd115-9738-45ec-9177-ba67e87b89b7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:39.575158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:39.616584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedd115-9738-45ec-9177-ba67e87b89b7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:39.658759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fb81c-9866-47f5-a525-00be7bea544b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:39.659312 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fb81c-9866-47f5-a525-00be7bea544b None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:39.659592 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fb81c-9866-47f5-a525-00be7bea544b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:39.659864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fb81c-9866-47f5-a525-00be7bea544b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:39.660528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fb81c-9866-47f5-a525-00be7bea544b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:39.661960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fb81c-9866-47f5-a525-00be7bea544b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:39.687335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 532/1059] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:01:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:41.096045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f9990-56b3-42a6-a7a9-f8b78d29b660 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:41.096646 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f9990-56b3-42a6-a7a9-f8b78d29b660 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:41.096646 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f9990-56b3-42a6-a7a9-f8b78d29b660 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:41.098176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 528/1060] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:41.103663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e2e29-5f04-4891-9f09-9e0950e180fd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:41.104142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e2e29-5f04-4891-9f09-9e0950e180fd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:41.104237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e2e29-5f04-4891-9f09-9e0950e180fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:41.124429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-376e2e29-5f04-4891-9f09-9e0950e180fd None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:41.125744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-376e2e29-5f04-4891-9f09-9e0950e180fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:41.208711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 533/1061] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:41.474158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f9990-56b3-42a6-a7a9-f8b78d29b660 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:41.563926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b6ce8-7063-4699-915d-702d09c4ffcd None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:41.564248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b6ce8-7063-4699-915d-702d09c4ffcd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:41.564500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b6ce8-7063-4699-915d-702d09c4ffcd None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:41.566052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 534/1062] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:01:41.571585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a4458-38c2-444d-9277-575b0227ee5d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:41.571830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a4458-38c2-444d-9277-575b0227ee5d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:41.572094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a4458-38c2-444d-9277-575b0227ee5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:41.588687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f9990-56b3-42a6-a7a9-f8b78d29b660 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:41.598837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3c3a4458-38c2-444d-9277-575b0227ee5d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:41.600034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3c3a4458-38c2-444d-9277-575b0227ee5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:41.632630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaf6ac-d8ca-4494-9acc-5b0e7e41bfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:41.633589 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaf6ac-d8ca-4494-9acc-5b0e7e41bfdf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:41.634010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaf6ac-d8ca-4494-9acc-5b0e7e41bfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:41.634440 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaf6ac-d8ca-4494-9acc-5b0e7e41bfdf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:41.646537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaf6ac-d8ca-4494-9acc-5b0e7e41bfdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f4aaf6ac-d8ca-4494-9acc-5b0e7e41bfdf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:41.652339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:41.653566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:41.654613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:41.654613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:41.654613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:41.654613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:41.654613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:41.654613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:41.654613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:41.654613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:41.654613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:41.654613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:41.654613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 21:01:41.654613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:41.655776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:41.658018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 529/1063] 10.222.0.79 () {56 vars in 1120 bytes} [Tue Sep 6 21:01:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:41.663715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaf6ac-d8ca-4494-9acc-5b0e7e41bfdf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:41.672929 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 535/1064] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:41.731888 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:41.803421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaf6ac-d8ca-4494-9acc-5b0e7e41bfdf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:41.841153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982b5f0-007e-4f3f-9424-49268e0a6350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:41.841767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982b5f0-007e-4f3f-9424-49268e0a6350 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:41.842041 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982b5f0-007e-4f3f-9424-49268e0a6350 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:41.842307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982b5f0-007e-4f3f-9424-49268e0a6350 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:41.842968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982b5f0-007e-4f3f-9424-49268e0a6350 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:41.843261 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982b5f0-007e-4f3f-9424-49268e0a6350 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:41.844630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982b5f0-007e-4f3f-9424-49268e0a6350 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:41.858545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 530/1065] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Sep 6 21:01:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:41.859848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a4458-38c2-444d-9277-575b0227ee5d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:41.899971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a6f00-537c-47fd-8700-5bdb2f4f695d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:41.900384 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a6f00-537c-47fd-8700-5bdb2f4f695d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:41.900783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a6f00-537c-47fd-8700-5bdb2f4f695d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:41.903280 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 531/1066] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:01:41.908298 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ac169-1ffb-4476-8899-815970b46f2a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:41.908573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ac169-1ffb-4476-8899-815970b46f2a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:41.908881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ac169-1ffb-4476-8899-815970b46f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:41.927436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f91ac169-1ffb-4476-8899-815970b46f2a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:41.928630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f91ac169-1ffb-4476-8899-815970b46f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:41.982367 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a4458-38c2-444d-9277-575b0227ee5d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:41.997874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 532/1067] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:41.998746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ac169-1ffb-4476-8899-815970b46f2a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:42.019786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb82b75-52d2-4127-b4cc-de14b7a53e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:42.020435 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb82b75-52d2-4127-b4cc-de14b7a53e2f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:42.020736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb82b75-52d2-4127-b4cc-de14b7a53e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:42.021040 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb82b75-52d2-4127-b4cc-de14b7a53e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:42.033123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb82b75-52d2-4127-b4cc-de14b7a53e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:42.035316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb82b75-52d2-4127-b4cc-de14b7a53e2f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7bb82b75-52d2-4127-b4cc-de14b7a53e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:42.052196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:42.053543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:42.054706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:42.054706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:42.054706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:42.054706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 536/1068] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:42.059627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb82b75-52d2-4127-b4cc-de14b7a53e2f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:42.060724 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:42.107749 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:42.136158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ac169-1ffb-4476-8899-815970b46f2a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:42.172844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5bfda-e92f-4159-86de-28b227c1e696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:42.173615 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5bfda-e92f-4159-86de-28b227c1e696 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:42.173747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5bfda-e92f-4159-86de-28b227c1e696 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:42.174003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5bfda-e92f-4159-86de-28b227c1e696 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:42.181775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb82b75-52d2-4127-b4cc-de14b7a53e2f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:42.186112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5bfda-e92f-4159-86de-28b227c1e696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f7d5bfda-e92f-4159-86de-28b227c1e696 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:42.190874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:42.191825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:42.192775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:42.192775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:42.192775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:42.192775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:42.192775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:42.192775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:42.192775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:42.192775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:42.192775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:42.192775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:42.192775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:01:42.192775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:42.196849 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 533/1069] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:01:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:42.198115 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5bfda-e92f-4159-86de-28b227c1e696 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:42.225672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948c539-b9f1-4422-bd9d-8bf738e667ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:42.226320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948c539-b9f1-4422-bd9d-8bf738e667ad None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:42.226575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948c539-b9f1-4422-bd9d-8bf738e667ad None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:42.226847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948c539-b9f1-4422-bd9d-8bf738e667ad None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:42.227495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948c539-b9f1-4422-bd9d-8bf738e667ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:42.227798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948c539-b9f1-4422-bd9d-8bf738e667ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:42.229223 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948c539-b9f1-4422-bd9d-8bf738e667ad None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:42.244612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 537/1070] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:42.247698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948c539-b9f1-4422-bd9d-8bf738e667ad None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:42.306955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5bfda-e92f-4159-86de-28b227c1e696 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:42.308630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:42.346794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a8402-2c92-49d1-8964-8dfb36c56324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:42.348308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a8402-2c92-49d1-8964-8dfb36c56324 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:42.348647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:42.349093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a8402-2c92-49d1-8964-8dfb36c56324 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:42.349787 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a8402-2c92-49d1-8964-8dfb36c56324 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:42.351193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a8402-2c92-49d1-8964-8dfb36c56324 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:42.351954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a8402-2c92-49d1-8964-8dfb36c56324 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:42.352708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948c539-b9f1-4422-bd9d-8bf738e667ad None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:42.355658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a8402-2c92-49d1-8964-8dfb36c56324 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:42.364256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 534/1071] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:01:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:42.365178 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a8402-2c92-49d1-8964-8dfb36c56324 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:42.389583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ac69a-f655-473f-a23d-031cb776d3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:42.390316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ac69a-f655-473f-a23d-031cb776d3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:42.390573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ac69a-f655-473f-a23d-031cb776d3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:42.390852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ac69a-f655-473f-a23d-031cb776d3d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:42.410151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ac69a-f655-473f-a23d-031cb776d3d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:42.411651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ac69a-f655-473f-a23d-031cb776d3d0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b51ac69a-f655-473f-a23d-031cb776d3d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:01:42.424723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:42.425802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:42.426778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:42.426778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:42.426778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:01:42.426778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:01:42.426778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:01:42.426778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:01:42.426778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:01:42.426778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:01:42.426778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:01:42.426778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:42.426778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 538/1072] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Sep 6 21:01:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:42.427391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ac69a-f655-473f-a23d-031cb776d3d0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:42.497200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a8402-2c92-49d1-8964-8dfb36c56324 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:42.531044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d42e00-8bb9-4074-97dc-690b4d039f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:42.531650 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d42e00-8bb9-4074-97dc-690b4d039f4b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:42.531933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d42e00-8bb9-4074-97dc-690b4d039f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:42.532193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d42e00-8bb9-4074-97dc-690b4d039f4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:42.537824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ac69a-f655-473f-a23d-031cb776d3d0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:42.565675 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:42.574887 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac9760-a568-4880-ab04-ecd64dc6ac03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:42.575524 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac9760-a568-4880-ab04-ecd64dc6ac03 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:42.575794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac9760-a568-4880-ab04-ecd64dc6ac03 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:42.576122 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac9760-a568-4880-ab04-ecd64dc6ac03 None admin] PATH_INFO: `/v3/users/3ac49e37b337489fa1ea68b29f2c548c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:42.577229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac9760-a568-4880-ab04-ecd64dc6ac03 None admin] RBAC: Authorizing `identity:get_user(user_id=3ac49e37b337489fa1ea68b29f2c548c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:42.578676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac9760-a568-4880-ab04-ecd64dc6ac03 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:42.580277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 539/1073] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:01:42 2022] GET /identity/v3/users/3ac49e37b337489fa1ea68b29f2c548c => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:42.581058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac9760-a568-4880-ab04-ecd64dc6ac03 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:42.612536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d42e00-8bb9-4074-97dc-690b4d039f4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:42.612799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:42.614075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d42e00-8bb9-4074-97dc-690b4d039f4b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:42.697292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac9760-a568-4880-ab04-ecd64dc6ac03 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:42.716068 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 535/1074] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:01:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:01:42.717094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d42e00-8bb9-4074-97dc-690b4d039f4b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:42.739217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62763d80-8091-434c-a72c-5f43b4758aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:42.739880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763d80-8091-434c-a72c-5f43b4758aaf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:42.740342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763d80-8091-434c-a72c-5f43b4758aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:42.740809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763d80-8091-434c-a72c-5f43b4758aaf None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:42.741690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62763d80-8091-434c-a72c-5f43b4758aaf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:42.741988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62763d80-8091-434c-a72c-5f43b4758aaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:42.743436 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62763d80-8091-434c-a72c-5f43b4758aaf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:42.762064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 540/1075] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Sep 6 21:01:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:42.762807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62763d80-8091-434c-a72c-5f43b4758aaf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:42.782127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:42.824811 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d42e00-8bb9-4074-97dc-690b4d039f4b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:42.861570 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:42.874671 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7defeee-01f4-4332-9c8e-c88f39ff6172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:42.875305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7defeee-01f4-4332-9c8e-c88f39ff6172 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:42.875586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7defeee-01f4-4332-9c8e-c88f39ff6172 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:42.875889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7defeee-01f4-4332-9c8e-c88f39ff6172 None admin] PATH_INFO: `/v3/projects/29972165e963424e832409079cb92fb2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:42.877171 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7defeee-01f4-4332-9c8e-c88f39ff6172 None admin] RBAC: Authorizing `identity:get_project(project_id=29972165e963424e832409079cb92fb2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:42.878620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7defeee-01f4-4332-9c8e-c88f39ff6172 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:42.879378 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62763d80-8091-434c-a72c-5f43b4758aaf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:42.880253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 536/1076] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:01:42 2022] GET /identity/v3/projects/29972165e963424e832409079cb92fb2 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:42.881042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7defeee-01f4-4332-9c8e-c88f39ff6172 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:42.916379 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753b422-c8d7-40ea-965f-f3a203b80edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:42.917094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753b422-c8d7-40ea-965f-f3a203b80edd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:42.917371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753b422-c8d7-40ea-965f-f3a203b80edd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:42.917637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753b422-c8d7-40ea-965f-f3a203b80edd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:42.928056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753b422-c8d7-40ea-965f-f3a203b80edd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:42.929553 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753b422-c8d7-40ea-965f-f3a203b80edd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2753b422-c8d7-40ea-965f-f3a203b80edd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:42.945487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:42.946658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:42.947563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:42.947563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:42.947563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:42.947563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 541/1077] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:42.947882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753b422-c8d7-40ea-965f-f3a203b80edd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:42.991073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7defeee-01f4-4332-9c8e-c88f39ff6172 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:43.020998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:43.028184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069a0a0-7c09-4203-a0ba-3601d30bfb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:43.028834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069a0a0-7c09-4203-a0ba-3601d30bfb9f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:43.029108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069a0a0-7c09-4203-a0ba-3601d30bfb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:43.029372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069a0a0-7c09-4203-a0ba-3601d30bfb9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:43.041220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069a0a0-7c09-4203-a0ba-3601d30bfb9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:43.042682 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069a0a0-7c09-4203-a0ba-3601d30bfb9f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0069a0a0-7c09-4203-a0ba-3601d30bfb9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:43.057398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:43.059218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:43.061656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:43.061656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:43.061656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:43.061656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 537/1078] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:43.061656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069a0a0-7c09-4203-a0ba-3601d30bfb9f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:43.065485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753b422-c8d7-40ea-965f-f3a203b80edd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:43.101448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2a008-b711-40d1-9b49-009b528b3dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:43.102526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2a008-b711-40d1-9b49-009b528b3dc3 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:43.102864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2a008-b711-40d1-9b49-009b528b3dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:43.103203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2a008-b711-40d1-9b49-009b528b3dc3 None admin] PATH_INFO: `/v3/projects/29972165e963424e832409079cb92fb2/users/3ac49e37b337489fa1ea68b29f2c548c/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:43.105144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2a008-b711-40d1-9b49-009b528b3dc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=29972165e963424e832409079cb92fb2, user_id=3ac49e37b337489fa1ea68b29f2c548c, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:43.106647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2a008-b711-40d1-9b49-009b528b3dc3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:43.109962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:43.124585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 542/1079] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:01:42 2022] PUT /identity/v3/projects/29972165e963424e832409079cb92fb2/users/3ac49e37b337489fa1ea68b29f2c548c/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:01:43.125420 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2a008-b711-40d1-9b49-009b528b3dc3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:43.181592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069a0a0-7c09-4203-a0ba-3601d30bfb9f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:43.228244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98175c99-0555-4c45-abc5-84e4733cb7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:43.229052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98175c99-0555-4c45-abc5-84e4733cb7cd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:43.229263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98175c99-0555-4c45-abc5-84e4733cb7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:43.229538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98175c99-0555-4c45-abc5-84e4733cb7cd None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:43.230205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98175c99-0555-4c45-abc5-84e4733cb7cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:43.230654 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98175c99-0555-4c45-abc5-84e4733cb7cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:43.232128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98175c99-0555-4c45-abc5-84e4733cb7cd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:43.245291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 538/1080] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:43.260518 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2a008-b711-40d1-9b49-009b528b3dc3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:43.309508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7769dcf-d215-428c-bf93-2fd75fe79d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:43.310405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7769dcf-d215-428c-bf93-2fd75fe79d4d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:43.310686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7769dcf-d215-428c-bf93-2fd75fe79d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:43.310952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7769dcf-d215-428c-bf93-2fd75fe79d4d None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:43.311637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7769dcf-d215-428c-bf93-2fd75fe79d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:43.311937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7769dcf-d215-428c-bf93-2fd75fe79d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:43.313523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7769dcf-d215-428c-bf93-2fd75fe79d4d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:43.329755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 543/1081] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:43.337548 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98175c99-0555-4c45-abc5-84e4733cb7cd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:43.396795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:43.468222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98175c99-0555-4c45-abc5-84e4733cb7cd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:43.506647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbca87-5e3d-4afb-a5dd-67d201a3814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:43.507291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbca87-5e3d-4afb-a5dd-67d201a3814e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:43.507569 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbca87-5e3d-4afb-a5dd-67d201a3814e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:43.507837 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbca87-5e3d-4afb-a5dd-67d201a3814e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:43.516200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbca87-5e3d-4afb-a5dd-67d201a3814e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:43.518138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbca87-5e3d-4afb-a5dd-67d201a3814e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-5cfbca87-5e3d-4afb-a5dd-67d201a3814e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:01:43.532493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:43.533571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:43.534656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:01:43.534656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:01:43.534656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:01:43.534656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:01:43.534656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:01:43.534656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:01:43.534656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:43.534656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:43.534656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 539/1082] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:43.544578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7769dcf-d215-428c-bf93-2fd75fe79d4d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:43.619066 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:43.673606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7769dcf-d215-428c-bf93-2fd75fe79d4d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:43.710540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b22ac0-0a27-4a4a-89c1-a699f5d45ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:43.711268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b22ac0-0a27-4a4a-89c1-a699f5d45ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:43.711552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b22ac0-0a27-4a4a-89c1-a699f5d45ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:43.711817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b22ac0-0a27-4a4a-89c1-a699f5d45ee6 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:43.712495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b22ac0-0a27-4a4a-89c1-a699f5d45ee6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:43.712829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b22ac0-0a27-4a4a-89c1-a699f5d45ee6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:43.714273 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b22ac0-0a27-4a4a-89c1-a699f5d45ee6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:43.727029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 544/1083] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:43.733022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbca87-5e3d-4afb-a5dd-67d201a3814e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:43.840383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbca87-5e3d-4afb-a5dd-67d201a3814e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:43.842711 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:43.876553 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b856521-286a-4313-b4b7-5d1d1f739ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:43.877242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b856521-286a-4313-b4b7-5d1d1f739ba5 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:43.877516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b856521-286a-4313-b4b7-5d1d1f739ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:43.877779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b856521-286a-4313-b4b7-5d1d1f739ba5 None admin] PATH_INFO: `/v3/projects/05af249ba0a047fe83aefb078e203e53/users/12d51454d20c46d7a5e6d963a2f987cc/roles/6bc0365fc21845598826c2848897e72b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:43.880113 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b856521-286a-4313-b4b7-5d1d1f739ba5 None admin] RBAC: Authorizing `identity:create_grant(project_id=05af249ba0a047fe83aefb078e203e53, user_id=12d51454d20c46d7a5e6d963a2f987cc, role_id=6bc0365fc21845598826c2848897e72b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:43.881587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b856521-286a-4313-b4b7-5d1d1f739ba5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:43.896593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 540/1084] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:01:43 2022] PUT /identity/v3/projects/05af249ba0a047fe83aefb078e203e53/users/12d51454d20c46d7a5e6d963a2f987cc/roles/6bc0365fc21845598826c2848897e72b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:01:45.313283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e379f-c5e3-4124-8c75-c45a099dc66c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:45.313861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e379f-c5e3-4124-8c75-c45a099dc66c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:45.313861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e379f-c5e3-4124-8c75-c45a099dc66c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:45.315270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 545/1085] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:45.320864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491c56e-485b-44eb-9c70-915a6b941eba None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:45.321369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491c56e-485b-44eb-9c70-915a6b941eba None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:45.321369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491c56e-485b-44eb-9c70-915a6b941eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:45.340057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e491c56e-485b-44eb-9c70-915a6b941eba None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:45.341282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e491c56e-485b-44eb-9c70-915a6b941eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:45.409970 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 541/1086] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:45.671373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e379f-c5e3-4124-8c75-c45a099dc66c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:45.793800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e379f-c5e3-4124-8c75-c45a099dc66c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:45.819588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:45.844995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd91fb-4f4a-456f-8ce8-631a3dae6890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:45.845551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd91fb-4f4a-456f-8ce8-631a3dae6890 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:45.845824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd91fb-4f4a-456f-8ce8-631a3dae6890 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:45.846100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd91fb-4f4a-456f-8ce8-631a3dae6890 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:45.856932 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd91fb-4f4a-456f-8ce8-631a3dae6890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-aecd91fb-4f4a-456f-8ce8-631a3dae6890 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:45.861795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:45.862673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:45.863515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:45.863515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:45.863515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:45.863515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:45.863515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:45.863515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:45.863515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:45.863515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:45.863515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:45.863515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:45.863515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:01:45.863515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:45.863515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 546/1087] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:01:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:45.869192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491c56e-485b-44eb-9c70-915a6b941eba None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:45.974512 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:45.982197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491c56e-485b-44eb-9c70-915a6b941eba None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:46.017305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3a7d6-c658-4edc-afaf-6c52f264eca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:46.017914 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3a7d6-c658-4edc-afaf-6c52f264eca6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:46.018191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3a7d6-c658-4edc-afaf-6c52f264eca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:46.018460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3a7d6-c658-4edc-afaf-6c52f264eca6 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:46.019123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3a7d6-c658-4edc-afaf-6c52f264eca6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:46.019422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3a7d6-c658-4edc-afaf-6c52f264eca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:46.021069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3a7d6-c658-4edc-afaf-6c52f264eca6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:46.029651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 542/1088] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:01:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:46.035653 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd91fb-4f4a-456f-8ce8-631a3dae6890 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:46.046685 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae8bb4-453f-41c8-870b-f41741b8ce7c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:46.047008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae8bb4-453f-41c8-870b-f41741b8ce7c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:46.047222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae8bb4-453f-41c8-870b-f41741b8ce7c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:46.048558 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 543/1089] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:01:46.054094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa530f46-45b1-4c4a-97a4-4e580378b091 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:46.054372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa530f46-45b1-4c4a-97a4-4e580378b091 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:46.054643 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa530f46-45b1-4c4a-97a4-4e580378b091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:46.075399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fa530f46-45b1-4c4a-97a4-4e580378b091 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:46.076622 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fa530f46-45b1-4c4a-97a4-4e580378b091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:46.140548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd91fb-4f4a-456f-8ce8-631a3dae6890 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:46.145991 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 544/1090] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:46.187782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61144e6a-b85a-429b-8de3-7cea7a9c059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:46.188426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61144e6a-b85a-429b-8de3-7cea7a9c059a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:46.188703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61144e6a-b85a-429b-8de3-7cea7a9c059a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:46.189004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61144e6a-b85a-429b-8de3-7cea7a9c059a None admin] PATH_INFO: `/v3/users/3ac49e37b337489fa1ea68b29f2c548c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:46.190177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61144e6a-b85a-429b-8de3-7cea7a9c059a None admin] RBAC: Authorizing `identity:get_user(user_id=3ac49e37b337489fa1ea68b29f2c548c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:46.191641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61144e6a-b85a-429b-8de3-7cea7a9c059a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:46.193442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 547/1091] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:01:46 2022] GET /identity/v3/users/3ac49e37b337489fa1ea68b29f2c548c => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:46.199728 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa530f46-45b1-4c4a-97a4-4e580378b091 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:46.289711 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:46.306729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa530f46-45b1-4c4a-97a4-4e580378b091 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:46.342986 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878342d-8a4b-41cc-9782-284680049d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:46.343641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878342d-8a4b-41cc-9782-284680049d00 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:46.343854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878342d-8a4b-41cc-9782-284680049d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:46.344124 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878342d-8a4b-41cc-9782-284680049d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:46.411590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61144e6a-b85a-429b-8de3-7cea7a9c059a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:46.412135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878342d-8a4b-41cc-9782-284680049d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:46.413219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878342d-8a4b-41cc-9782-284680049d00 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:46.463919 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:46.520219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61144e6a-b85a-429b-8de3-7cea7a9c059a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:46.520920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:46.538468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 545/1092] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:01:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:01:46.544416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878342d-8a4b-41cc-9782-284680049d00 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:46.565405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af11b07-c84f-48f9-af69-658e0b8427fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:46.566140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af11b07-c84f-48f9-af69-658e0b8427fb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:46.566326 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af11b07-c84f-48f9-af69-658e0b8427fb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:46.566591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af11b07-c84f-48f9-af69-658e0b8427fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:46.577673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af11b07-c84f-48f9-af69-658e0b8427fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2af11b07-c84f-48f9-af69-658e0b8427fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:01:46.585967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:46.587468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:46.588309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:46.588309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:46.588309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:46.588309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:46.588309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:01:46.588309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:01:46.588309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:01:46.588309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:01:46.588309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:01:46.588309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:01:46.588309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 21:01:46.588309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:46.588309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 548/1093] 10.222.0.79 () {56 vars in 1120 bytes} [Tue Sep 6 21:01:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:46.594846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af11b07-c84f-48f9-af69-658e0b8427fb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:46.657029 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878342d-8a4b-41cc-9782-284680049d00 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:46.693260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab233b5-9fc9-41fe-994e-0bef763aaa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:46.693844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab233b5-9fc9-41fe-994e-0bef763aaa37 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:46.694121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab233b5-9fc9-41fe-994e-0bef763aaa37 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:46.694392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab233b5-9fc9-41fe-994e-0bef763aaa37 None admin] PATH_INFO: `/v3/projects/29972165e963424e832409079cb92fb2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:46.695535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab233b5-9fc9-41fe-994e-0bef763aaa37 None admin] RBAC: Authorizing `identity:get_project(project_id=29972165e963424e832409079cb92fb2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:46.697019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab233b5-9fc9-41fe-994e-0bef763aaa37 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:46.698678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 546/1094] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:01:46 2022] GET /identity/v3/projects/29972165e963424e832409079cb92fb2 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:46.699454 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af11b07-c84f-48f9-af69-658e0b8427fb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:46.702007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:46.705059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab233b5-9fc9-41fe-994e-0bef763aaa37 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:46.736626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa64678-0653-48d8-bc5e-a10138b48197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:46.737064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa64678-0653-48d8-bc5e-a10138b48197 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:46.737348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa64678-0653-48d8-bc5e-a10138b48197 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:46.737612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa64678-0653-48d8-bc5e-a10138b48197 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:46.738285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa64678-0653-48d8-bc5e-a10138b48197 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:46.738585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa64678-0653-48d8-bc5e-a10138b48197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:46.739978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa64678-0653-48d8-bc5e-a10138b48197 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:46.748505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 549/1095] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Sep 6 21:01:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:46.754369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa64678-0653-48d8-bc5e-a10138b48197 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:46.761827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:46.811778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab233b5-9fc9-41fe-994e-0bef763aaa37 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:46.853005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d47a9-0892-4a97-a2ce-e25903eca00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:46.853636 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d47a9-0892-4a97-a2ce-e25903eca00e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:46.853909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d47a9-0892-4a97-a2ce-e25903eca00e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:46.854175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d47a9-0892-4a97-a2ce-e25903eca00e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:46.854895 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d47a9-0892-4a97-a2ce-e25903eca00e None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=29972165e963424e832409079cb92fb2, user.id=3ac49e37b337489fa1ea68b29f2c548c) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:46.855195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d47a9-0892-4a97-a2ce-e25903eca00e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:46.855327 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa64678-0653-48d8-bc5e-a10138b48197 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:46.856720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d47a9-0892-4a97-a2ce-e25903eca00e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:46.862608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 547/1096] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:01:46 2022] GET /identity/v3/role_assignments?user.id=3ac49e37b337489fa1ea68b29f2c548c&scope.project.id=29972165e963424e832409079cb92fb2&role.id=3b30504991194002b41f8587d0851c10 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:46.900690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6a090-f172-43e7-b4c2-13082f523406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:46.901377 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6a090-f172-43e7-b4c2-13082f523406 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:46.901651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6a090-f172-43e7-b4c2-13082f523406 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:46.901907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6a090-f172-43e7-b4c2-13082f523406 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:46.914320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6a090-f172-43e7-b4c2-13082f523406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:46.915800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6a090-f172-43e7-b4c2-13082f523406 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-31b6a090-f172-43e7-b4c2-13082f523406 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:46.932578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:46.933729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:46.934639 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:46.934639 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:46.934639 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:46.934639 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 550/1097] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:46.941019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d47a9-0892-4a97-a2ce-e25903eca00e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:47.060899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d47a9-0892-4a97-a2ce-e25903eca00e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:47.079715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:47.100484 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7c9b4-3d7b-4d95-9588-793de707ad0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:47.101146 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7c9b4-3d7b-4d95-9588-793de707ad0f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:47.101424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7c9b4-3d7b-4d95-9588-793de707ad0f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:47.101690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7c9b4-3d7b-4d95-9588-793de707ad0f None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:47.102330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7c9b4-3d7b-4d95-9588-793de707ad0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:47.102627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7c9b4-3d7b-4d95-9588-793de707ad0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:47.103988 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7c9b4-3d7b-4d95-9588-793de707ad0f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:47.116215 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 548/1098] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:47.122083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6a090-f172-43e7-b4c2-13082f523406 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:47.143167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:47.229167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6a090-f172-43e7-b4c2-13082f523406 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:47.264291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fb3f7-c7eb-42cd-8496-bd2a1c97acf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:47.264956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fb3f7-c7eb-42cd-8496-bd2a1c97acf7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:47.265233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fb3f7-c7eb-42cd-8496-bd2a1c97acf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:47.265506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fb3f7-c7eb-42cd-8496-bd2a1c97acf7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:47.273564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fb3f7-c7eb-42cd-8496-bd2a1c97acf7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:47.275024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fb3f7-c7eb-42cd-8496-bd2a1c97acf7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-537fb3f7-c7eb-42cd-8496-bd2a1c97acf7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:01:47.286657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:47.287744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:47.294005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:47.294005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:47.294005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:01:47.294005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:01:47.294005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:01:47.294005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:01:47.294005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:01:47.294005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:01:47.294005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 21:01:47.294005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:01:47.294005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 551/1099] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Sep 6 21:01:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:01:47.294603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7c9b4-3d7b-4d95-9588-793de707ad0f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:47.398424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7c9b4-3d7b-4d95-9588-793de707ad0f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:47.437665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a59ef-3864-4538-872c-cce2fd18a498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:47.438267 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a59ef-3864-4538-872c-cce2fd18a498 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:47.438546 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a59ef-3864-4538-872c-cce2fd18a498 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:47.438813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a59ef-3864-4538-872c-cce2fd18a498 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:47.439521 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a59ef-3864-4538-872c-cce2fd18a498 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:47.439843 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a59ef-3864-4538-872c-cce2fd18a498 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:47.441323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a59ef-3864-4538-872c-cce2fd18a498 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:47.459247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 549/1100] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Sep 6 21:01:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:47.465395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fb3f7-c7eb-42cd-8496-bd2a1c97acf7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:47.550375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:47.576628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fb3f7-c7eb-42cd-8496-bd2a1c97acf7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:47.616615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b16ea-8a5b-438d-a341-2db7c3daa34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:47.617253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b16ea-8a5b-438d-a341-2db7c3daa34f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:47.617529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b16ea-8a5b-438d-a341-2db7c3daa34f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:47.617796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b16ea-8a5b-438d-a341-2db7c3daa34f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:47.688945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b16ea-8a5b-438d-a341-2db7c3daa34f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:47.690342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b16ea-8a5b-438d-a341-2db7c3daa34f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:47.768180 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:47.798409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 552/1101] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:01:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:01:47.799706 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a59ef-3864-4538-872c-cce2fd18a498 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:47.838409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:47.904264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a59ef-3864-4538-872c-cce2fd18a498 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:47.940988 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5aa98-6823-4ac5-82fc-002858716fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:47.941549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5aa98-6823-4ac5-82fc-002858716fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:47.941822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5aa98-6823-4ac5-82fc-002858716fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:47.942085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5aa98-6823-4ac5-82fc-002858716fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:47.953065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5aa98-6823-4ac5-82fc-002858716fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:47.954506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5aa98-6823-4ac5-82fc-002858716fd8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-aea5aa98-6823-4ac5-82fc-002858716fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:01:47.969057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:47.970153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:01:47.971454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:01:47.971454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:01:47.971454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:47.971454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 550/1102] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:47.976117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b16ea-8a5b-438d-a341-2db7c3daa34f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:48.081597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b16ea-8a5b-438d-a341-2db7c3daa34f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:48.119718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34d9a5-39c5-4b9c-b093-7d8633a67fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:48.120330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34d9a5-39c5-4b9c-b093-7d8633a67fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:48.120606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34d9a5-39c5-4b9c-b093-7d8633a67fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:48.120901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34d9a5-39c5-4b9c-b093-7d8633a67fc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:48.121601 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34d9a5-39c5-4b9c-b093-7d8633a67fc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:48.121899 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34d9a5-39c5-4b9c-b093-7d8633a67fc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:48.123308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34d9a5-39c5-4b9c-b093-7d8633a67fc8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:48.135107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 553/1103] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:48.140821 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5aa98-6823-4ac5-82fc-002858716fd8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:48.230713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:48.244020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5aa98-6823-4ac5-82fc-002858716fd8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:48.278714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21db82a-54b6-4266-b59d-a4c6ffffdcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:48.279347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21db82a-54b6-4266-b59d-a4c6ffffdcfa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:48.279623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21db82a-54b6-4266-b59d-a4c6ffffdcfa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:48.279895 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21db82a-54b6-4266-b59d-a4c6ffffdcfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:48.286664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21db82a-54b6-4266-b59d-a4c6ffffdcfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:48.288145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21db82a-54b6-4266-b59d-a4c6ffffdcfa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d21db82a-54b6-4266-b59d-a4c6ffffdcfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:01:48.299571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:01:48.300554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:01:48.301500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:01:48.301500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:01:48.301500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:01:48.301500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:01:48.301500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:01:48.301500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:01:48.301500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:01:48.301500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:01:48.301500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 551/1104] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:01:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:01:48.306797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34d9a5-39c5-4b9c-b093-7d8633a67fc8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:48.349752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:48.410814 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34d9a5-39c5-4b9c-b093-7d8633a67fc8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:48.446182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258dd69-221a-4032-906a-288ca9b8c4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:48.446803 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258dd69-221a-4032-906a-288ca9b8c4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:48.447080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258dd69-221a-4032-906a-288ca9b8c4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:48.447351 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258dd69-221a-4032-906a-288ca9b8c4b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:48.448049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258dd69-221a-4032-906a-288ca9b8c4b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:48.448353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258dd69-221a-4032-906a-288ca9b8c4b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:48.449985 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258dd69-221a-4032-906a-288ca9b8c4b1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:48.462758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 554/1105] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:01:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:48.469616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21db82a-54b6-4266-b59d-a4c6ffffdcfa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:48.575779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21db82a-54b6-4266-b59d-a4c6ffffdcfa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:48.626459 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3a853-93b4-4b8e-ac58-f5995e90c67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:48.627059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3a853-93b4-4b8e-ac58-f5995e90c67b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:48.627332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3a853-93b4-4b8e-ac58-f5995e90c67b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:48.627601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3a853-93b4-4b8e-ac58-f5995e90c67b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:48.628324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3a853-93b4-4b8e-ac58-f5995e90c67b None admin] RBAC: Adding query filter params (role.id=6bc0365fc21845598826c2848897e72b, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=12d51454d20c46d7a5e6d963a2f987cc) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:48.628624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3a853-93b4-4b8e-ac58-f5995e90c67b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:48.630088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3a853-93b4-4b8e-ac58-f5995e90c67b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:48.637483 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 552/1106] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:01:48 2022] GET /identity/v3/role_assignments?user.id=12d51454d20c46d7a5e6d963a2f987cc&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=6bc0365fc21845598826c2848897e72b => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:49.059559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc69eee-22a4-423e-837c-537e790e0d7c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:49.061016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc69eee-22a4-423e-837c-537e790e0d7c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:49.061016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc69eee-22a4-423e-837c-537e790e0d7c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:49.061841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 555/1107] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:49.067559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ab1b5-2208-441f-bca4-6e9e75335491 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:49.067878 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ab1b5-2208-441f-bca4-6e9e75335491 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:49.068113 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ab1b5-2208-441f-bca4-6e9e75335491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:49.087830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c06ab1b5-2208-441f-bca4-6e9e75335491 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:49.089137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c06ab1b5-2208-441f-bca4-6e9e75335491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:49.097582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:49.162740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 553/1108] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:49.437015 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc69eee-22a4-423e-837c-537e790e0d7c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:49.538080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:49.543711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc69eee-22a4-423e-837c-537e790e0d7c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:49.581833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de5940-2e32-4f11-8a48-541a3973ace6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:49.582576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de5940-2e32-4f11-8a48-541a3973ace6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:49.582864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de5940-2e32-4f11-8a48-541a3973ace6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:49.583137 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de5940-2e32-4f11-8a48-541a3973ace6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:49.584392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de5940-2e32-4f11-8a48-541a3973ace6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:49.586342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de5940-2e32-4f11-8a48-541a3973ace6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:49.587667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 556/1109] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:01:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:49.593718 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ab1b5-2208-441f-bca4-6e9e75335491 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:49.703348 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ab1b5-2208-441f-bca4-6e9e75335491 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:49.737248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03180e66-7c76-406a-b135-a6f076f2c2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:49.737851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03180e66-7c76-406a-b135-a6f076f2c2cd None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:49.738132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03180e66-7c76-406a-b135-a6f076f2c2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:49.738400 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03180e66-7c76-406a-b135-a6f076f2c2cd None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:49.739055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03180e66-7c76-406a-b135-a6f076f2c2cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:49.740488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03180e66-7c76-406a-b135-a6f076f2c2cd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:49.758821 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:49.765991 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 554/1110] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 21:01:49 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:01:51.962261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f073496-057d-4b4c-8e54-5eb7fd9da753 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:51.962798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f073496-057d-4b4c-8e54-5eb7fd9da753 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:51.962798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f073496-057d-4b4c-8e54-5eb7fd9da753 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:51.964025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 557/1111] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:51.969798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613168c-370c-4402-8ccc-40087ad5593b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:51.970272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613168c-370c-4402-8ccc-40087ad5593b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:51.970362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613168c-370c-4402-8ccc-40087ad5593b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:51.989885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5613168c-370c-4402-8ccc-40087ad5593b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:51.991106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5613168c-370c-4402-8ccc-40087ad5593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:52.071209 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 555/1112] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:01:52.332940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f073496-057d-4b4c-8e54-5eb7fd9da753 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:52.459182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f073496-057d-4b4c-8e54-5eb7fd9da753 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:52.497712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80982b2d-5e15-4f57-974a-147409c62a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:52.498364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80982b2d-5e15-4f57-974a-147409c62a36 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:52.498643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80982b2d-5e15-4f57-974a-147409c62a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:52.498910 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80982b2d-5e15-4f57-974a-147409c62a36 None admin] PATH_INFO: `/v3/roles/e945f087a5bc4c38bd8848782f76c214` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:52.500466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80982b2d-5e15-4f57-974a-147409c62a36 None admin] RBAC: Authorizing `identity:get_role(role_id=e945f087a5bc4c38bd8848782f76c214)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:52.501908 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80982b2d-5e15-4f57-974a-147409c62a36 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:52.503249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 558/1113] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:01:52 2022] GET /identity/v3/roles/e945f087a5bc4c38bd8848782f76c214 => generated 223 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:52.509516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613168c-370c-4402-8ccc-40087ad5593b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:52.615825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613168c-370c-4402-8ccc-40087ad5593b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:52.643787 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:52.656075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0bf42-dd12-413a-8e38-99e1bc02299b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:52.656743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0bf42-dd12-413a-8e38-99e1bc02299b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:52.657019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0bf42-dd12-413a-8e38-99e1bc02299b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:52.657293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0bf42-dd12-413a-8e38-99e1bc02299b None admin] PATH_INFO: `/v3/users/4b114938f0c340838ed584b3dd0e91d3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:52.676340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0bf42-dd12-413a-8e38-99e1bc02299b None admin] RBAC: Authorizing `identity:get_user(user_id=4b114938f0c340838ed584b3dd0e91d3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:52.678013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0bf42-dd12-413a-8e38-99e1bc02299b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:52.680626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 556/1114] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:01:52 2022] GET /identity/v3/users/4b114938f0c340838ed584b3dd0e91d3 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:52.688510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80982b2d-5e15-4f57-974a-147409c62a36 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:52.727501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:52.800775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80982b2d-5e15-4f57-974a-147409c62a36 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:52.843471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd33bf-d8e5-45d7-8b4c-60eb64968f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:52.844007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd33bf-d8e5-45d7-8b4c-60eb64968f29 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:52.844279 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd33bf-d8e5-45d7-8b4c-60eb64968f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:52.844566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd33bf-d8e5-45d7-8b4c-60eb64968f29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:52.913966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd33bf-d8e5-45d7-8b4c-60eb64968f29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:52.915731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd33bf-d8e5-45d7-8b4c-60eb64968f29 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:52.953145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:53.027940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 559/1115] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:01:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:01:53.034659 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0bf42-dd12-413a-8e38-99e1bc02299b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:53.155420 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0bf42-dd12-413a-8e38-99e1bc02299b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:53.191331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edeaf2b-0cc8-4556-b40f-7d5a317d6f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:53.191943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edeaf2b-0cc8-4556-b40f-7d5a317d6f66 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:53.192217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edeaf2b-0cc8-4556-b40f-7d5a317d6f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:53.192481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edeaf2b-0cc8-4556-b40f-7d5a317d6f66 None admin] PATH_INFO: `/v3/projects/29972165e963424e832409079cb92fb2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:53.193662 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edeaf2b-0cc8-4556-b40f-7d5a317d6f66 None admin] RBAC: Authorizing `identity:get_project(project_id=29972165e963424e832409079cb92fb2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:53.195092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edeaf2b-0cc8-4556-b40f-7d5a317d6f66 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:53.196780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 557/1116] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:01:53 2022] GET /identity/v3/projects/29972165e963424e832409079cb92fb2 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:53.203721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd33bf-d8e5-45d7-8b4c-60eb64968f29 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:53.313039 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd33bf-d8e5-45d7-8b4c-60eb64968f29 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:53.350346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf739b0b-149a-4e1e-9f49-69bbc612e434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:53.350948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf739b0b-149a-4e1e-9f49-69bbc612e434 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:53.351229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf739b0b-149a-4e1e-9f49-69bbc612e434 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:53.351501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf739b0b-149a-4e1e-9f49-69bbc612e434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:53.352219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf739b0b-149a-4e1e-9f49-69bbc612e434 None admin] RBAC: Adding query filter params (role.id=e945f087a5bc4c38bd8848782f76c214, scope.project.id=29972165e963424e832409079cb92fb2, user.id=4b114938f0c340838ed584b3dd0e91d3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:01:53.352524 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf739b0b-149a-4e1e-9f49-69bbc612e434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:53.354000 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf739b0b-149a-4e1e-9f49-69bbc612e434 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:53.355899 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:53.360556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 560/1117] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:01:53 2022] GET /identity/v3/role_assignments?user.id=4b114938f0c340838ed584b3dd0e91d3&scope.project.id=29972165e963424e832409079cb92fb2&role.id=e945f087a5bc4c38bd8848782f76c214 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:55.500307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde85dc-9335-455d-9818-d1b4ff0aa3df None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:55.501244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde85dc-9335-455d-9818-d1b4ff0aa3df None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:55.501244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde85dc-9335-455d-9818-d1b4ff0aa3df None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:55.503373 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 558/1118] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:55.509817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8316e-6a20-4f1c-aa17-2155570b12a8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:55.510300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8316e-6a20-4f1c-aa17-2155570b12a8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:55.510682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8316e-6a20-4f1c-aa17-2155570b12a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:55.536779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-06d8316e-6a20-4f1c-aa17-2155570b12a8 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:55.537991 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-06d8316e-6a20-4f1c-aa17-2155570b12a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:55.609094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 561/1119] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:55.880273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde85dc-9335-455d-9818-d1b4ff0aa3df None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:55.943357 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:55.992107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde85dc-9335-455d-9818-d1b4ff0aa3df None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:56.027495 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fa03b-01d8-435b-9015-23873d5a168f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:56.028147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fa03b-01d8-435b-9015-23873d5a168f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:56.028423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fa03b-01d8-435b-9015-23873d5a168f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:56.028700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fa03b-01d8-435b-9015-23873d5a168f None admin] PATH_INFO: `/v3/roles/e945f087a5bc4c38bd8848782f76c214` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:56.030149 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98fa03b-01d8-435b-9015-23873d5a168f None admin] RBAC: Authorizing `identity:get_role(role_id=e945f087a5bc4c38bd8848782f76c214)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:56.031529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98fa03b-01d8-435b-9015-23873d5a168f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:56.032904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 559/1120] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:01:55 2022] GET /identity/v3/roles/e945f087a5bc4c38bd8848782f76c214 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:56.039269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8316e-6a20-4f1c-aa17-2155570b12a8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:56.145976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8316e-6a20-4f1c-aa17-2155570b12a8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:56.174508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:56.183752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96206636-efef-4d2a-b320-aaf6784dd906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:56.184391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96206636-efef-4d2a-b320-aaf6784dd906 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:56.185061 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96206636-efef-4d2a-b320-aaf6784dd906 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:56.185330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96206636-efef-4d2a-b320-aaf6784dd906 None admin] PATH_INFO: `/v3/users/4b114938f0c340838ed584b3dd0e91d3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:56.186614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96206636-efef-4d2a-b320-aaf6784dd906 None admin] RBAC: Authorizing `identity:get_user(user_id=4b114938f0c340838ed584b3dd0e91d3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:56.188087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96206636-efef-4d2a-b320-aaf6784dd906 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:56.189723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 562/1121] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:01:56 2022] GET /identity/v3/users/4b114938f0c340838ed584b3dd0e91d3 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:56.196225 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fa03b-01d8-435b-9015-23873d5a168f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:56.303937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fa03b-01d8-435b-9015-23873d5a168f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:56.327124 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:56.340325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02765619-04fc-45f5-8c5a-574f36cedf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:56.340988 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02765619-04fc-45f5-8c5a-574f36cedf2b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:56.341269 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02765619-04fc-45f5-8c5a-574f36cedf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:56.341529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02765619-04fc-45f5-8c5a-574f36cedf2b None admin] PATH_INFO: `/v3/projects/29972165e963424e832409079cb92fb2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:56.342812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02765619-04fc-45f5-8c5a-574f36cedf2b None admin] RBAC: Authorizing `identity:get_project(project_id=29972165e963424e832409079cb92fb2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:56.344271 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02765619-04fc-45f5-8c5a-574f36cedf2b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:56.345947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 560/1122] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:01:56 2022] GET /identity/v3/projects/29972165e963424e832409079cb92fb2 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:56.352231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96206636-efef-4d2a-b320-aaf6784dd906 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:56.461011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96206636-efef-4d2a-b320-aaf6784dd906 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:56.496347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b1876-a3b4-43cf-8860-ae8a4875cb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:56.497021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b1876-a3b4-43cf-8860-ae8a4875cb2a None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:56.497289 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b1876-a3b4-43cf-8860-ae8a4875cb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:56.497564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b1876-a3b4-43cf-8860-ae8a4875cb2a None admin] PATH_INFO: `/v3/projects/29972165e963424e832409079cb92fb2/users/4b114938f0c340838ed584b3dd0e91d3/roles/e945f087a5bc4c38bd8848782f76c214` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:56.500175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b1876-a3b4-43cf-8860-ae8a4875cb2a None admin] RBAC: Authorizing `identity:create_grant(project_id=29972165e963424e832409079cb92fb2, user_id=4b114938f0c340838ed584b3dd0e91d3, role_id=e945f087a5bc4c38bd8848782f76c214)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:56.501650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b1876-a3b4-43cf-8860-ae8a4875cb2a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:56.518917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 563/1123] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:01:56 2022] PUT /identity/v3/projects/29972165e963424e832409079cb92fb2/users/4b114938f0c340838ed584b3dd0e91d3/roles/e945f087a5bc4c38bd8848782f76c214 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:01:58.652315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe8b02-3e74-4a64-a1fb-ff0aa0e3494d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:58.652891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe8b02-3e74-4a64-a1fb-ff0aa0e3494d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:58.652891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe8b02-3e74-4a64-a1fb-ff0aa0e3494d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:58.654265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 561/1124] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:01:58.660807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdb947-55db-432c-a5dd-f4d124bdbf14 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:58.661517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdb947-55db-432c-a5dd-f4d124bdbf14 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:58.661661 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdb947-55db-432c-a5dd-f4d124bdbf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:58.689816 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d8cdb947-55db-432c-a5dd-f4d124bdbf14 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:01:58.691027 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d8cdb947-55db-432c-a5dd-f4d124bdbf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:01:58.742893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:58.768682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 564/1125] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:01:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:01:59.038839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe8b02-3e74-4a64-a1fb-ff0aa0e3494d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:59.146564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe8b02-3e74-4a64-a1fb-ff0aa0e3494d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:59.181712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbe3a3-5c07-4e70-a518-ec566dce04cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:59.182321 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbe3a3-5c07-4e70-a518-ec566dce04cf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:59.182595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbe3a3-5c07-4e70-a518-ec566dce04cf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:59.182860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbe3a3-5c07-4e70-a518-ec566dce04cf None admin] PATH_INFO: `/v3/roles/e945f087a5bc4c38bd8848782f76c214` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:59.184328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbe3a3-5c07-4e70-a518-ec566dce04cf None admin] RBAC: Authorizing `identity:get_role(role_id=e945f087a5bc4c38bd8848782f76c214)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:59.186243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbe3a3-5c07-4e70-a518-ec566dce04cf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:59.187573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 562/1126] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:01:59 2022] GET /identity/v3/roles/e945f087a5bc4c38bd8848782f76c214 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:59.197218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdb947-55db-432c-a5dd-f4d124bdbf14 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:59.308973 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdb947-55db-432c-a5dd-f4d124bdbf14 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:59.345025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3462f-f100-4a98-a124-ed7d7e578b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:59.345649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3462f-f100-4a98-a124-ed7d7e578b7f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:59.345921 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3462f-f100-4a98-a124-ed7d7e578b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:59.346187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3462f-f100-4a98-a124-ed7d7e578b7f None admin] PATH_INFO: `/v3/users/4b114938f0c340838ed584b3dd0e91d3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:59.347468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3462f-f100-4a98-a124-ed7d7e578b7f None admin] RBAC: Authorizing `identity:get_user(user_id=4b114938f0c340838ed584b3dd0e91d3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:59.348947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3462f-f100-4a98-a124-ed7d7e578b7f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:59.350526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 565/1127] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:01:59 2022] GET /identity/v3/users/4b114938f0c340838ed584b3dd0e91d3 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:01:59.357202 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbe3a3-5c07-4e70-a518-ec566dce04cf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:59.431022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:59.481325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbe3a3-5c07-4e70-a518-ec566dce04cf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:59.516913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83fcf1-7308-4bb9-97bf-f28f6342d023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:59.517523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83fcf1-7308-4bb9-97bf-f28f6342d023 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:59.517797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83fcf1-7308-4bb9-97bf-f28f6342d023 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:59.518059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83fcf1-7308-4bb9-97bf-f28f6342d023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:59.595672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83fcf1-7308-4bb9-97bf-f28f6342d023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:59.597214 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83fcf1-7308-4bb9-97bf-f28f6342d023 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:59.660394 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:59.706010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 563/1128] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:01:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:01:59.715529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3462f-f100-4a98-a124-ed7d7e578b7f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:59.779769 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:01:59.835508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3462f-f100-4a98-a124-ed7d7e578b7f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:01:59.874587 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749fa99-de66-4e65-bddb-ed19019d0da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:01:59.875146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749fa99-de66-4e65-bddb-ed19019d0da3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:01:59.875428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749fa99-de66-4e65-bddb-ed19019d0da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:01:59.875696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749fa99-de66-4e65-bddb-ed19019d0da3 None admin] PATH_INFO: `/v3/projects/29972165e963424e832409079cb92fb2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:01:59.876978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749fa99-de66-4e65-bddb-ed19019d0da3 None admin] RBAC: Authorizing `identity:get_project(project_id=29972165e963424e832409079cb92fb2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:01:59.878429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749fa99-de66-4e65-bddb-ed19019d0da3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:01:59.880072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 566/1129] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:01:59 2022] GET /identity/v3/projects/29972165e963424e832409079cb92fb2 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:01:59.889844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83fcf1-7308-4bb9-97bf-f28f6342d023 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:01:59.995223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83fcf1-7308-4bb9-97bf-f28f6342d023 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:00.033207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcf5ca-efff-410e-8863-7b8d5c21bbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:00.033795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcf5ca-efff-410e-8863-7b8d5c21bbbc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:00.034069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcf5ca-efff-410e-8863-7b8d5c21bbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:00.034333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcf5ca-efff-410e-8863-7b8d5c21bbbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:00.035051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcf5ca-efff-410e-8863-7b8d5c21bbbc None admin] RBAC: Adding query filter params (role.id=e945f087a5bc4c38bd8848782f76c214, scope.project.id=29972165e963424e832409079cb92fb2, user.id=4b114938f0c340838ed584b3dd0e91d3) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:02:00.035352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcf5ca-efff-410e-8863-7b8d5c21bbbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:00.036808 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcf5ca-efff-410e-8863-7b8d5c21bbbc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:00.042599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 564/1130] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:01:59 2022] GET /identity/v3/role_assignments?user.id=4b114938f0c340838ed584b3dd0e91d3&scope.project.id=29972165e963424e832409079cb92fb2&role.id=e945f087a5bc4c38bd8848782f76c214 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:02.224737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bab27c-f93e-4cbe-8542-93593eee4fd7 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:02.225648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bab27c-f93e-4cbe-8542-93593eee4fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:02.225648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bab27c-f93e-4cbe-8542-93593eee4fd7 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:02.227594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 567/1131] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:02:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:02:02.235931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef678dd-73c9-4f7a-895e-a1db968f8cc8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:02.236395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef678dd-73c9-4f7a-895e-a1db968f8cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:02.236506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef678dd-73c9-4f7a-895e-a1db968f8cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:02.256417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2ef678dd-73c9-4f7a-895e-a1db968f8cc8 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:02:02.257657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2ef678dd-73c9-4f7a-895e-a1db968f8cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:02:02.275549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:02.328902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 565/1132] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:02:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:02:02.592736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bab27c-f93e-4cbe-8542-93593eee4fd7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:02.711336 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bab27c-f93e-4cbe-8542-93593eee4fd7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:02.736859 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:02.756396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac374b-b76e-476a-8c04-566ad15c6296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:02.757384 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac374b-b76e-476a-8c04-566ad15c6296 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:02.757767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac374b-b76e-476a-8c04-566ad15c6296 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:02.758152 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac374b-b76e-476a-8c04-566ad15c6296 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:02.759952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac374b-b76e-476a-8c04-566ad15c6296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:02.762181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac374b-b76e-476a-8c04-566ad15c6296 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:02.764393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 568/1133] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:02:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:02.770486 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef678dd-73c9-4f7a-895e-a1db968f8cc8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:02.877734 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef678dd-73c9-4f7a-895e-a1db968f8cc8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:02.918723 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf75dd-0842-4256-b83c-ebf765146de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:02.919579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf75dd-0842-4256-b83c-ebf765146de4 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:02.919933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf75dd-0842-4256-b83c-ebf765146de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:02.920293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf75dd-0842-4256-b83c-ebf765146de4 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:02.921309 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf75dd-0842-4256-b83c-ebf765146de4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:02.923550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf75dd-0842-4256-b83c-ebf765146de4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:02.959300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 566/1134] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 21:02:02 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:02:05.146281 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de817b1-6bbd-4307-8bf1-4c14401bbf06 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:05.147138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de817b1-6bbd-4307-8bf1-4c14401bbf06 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:05.147138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de817b1-6bbd-4307-8bf1-4c14401bbf06 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:05.149702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 569/1135] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:02:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:02:05.158392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bc910-b24d-4f7e-8ac6-2d5aeb6d55f8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:05.158926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bc910-b24d-4f7e-8ac6-2d5aeb6d55f8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:05.159105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bc910-b24d-4f7e-8ac6-2d5aeb6d55f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:05.171193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:05.187085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-876bc910-b24d-4f7e-8ac6-2d5aeb6d55f8 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:02:05.188288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-876bc910-b24d-4f7e-8ac6-2d5aeb6d55f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:02:05.264670 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 567/1136] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:02:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:02:05.535072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de817b1-6bbd-4307-8bf1-4c14401bbf06 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:05.652624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de817b1-6bbd-4307-8bf1-4c14401bbf06 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:05.688402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acedbe2-85c1-4bcf-b35a-4602cb4ee382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:05.689032 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acedbe2-85c1-4bcf-b35a-4602cb4ee382 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:05.689545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acedbe2-85c1-4bcf-b35a-4602cb4ee382 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:05.689660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acedbe2-85c1-4bcf-b35a-4602cb4ee382 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:05.690924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acedbe2-85c1-4bcf-b35a-4602cb4ee382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:05.692352 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acedbe2-85c1-4bcf-b35a-4602cb4ee382 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:05.693962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 570/1137] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:02:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:05.702940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bc910-b24d-4f7e-8ac6-2d5aeb6d55f8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:05.816847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bc910-b24d-4f7e-8ac6-2d5aeb6d55f8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:05.851818 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab11bd-4a2c-4216-99ed-d7f2f66dc548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:05.852474 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab11bd-4a2c-4216-99ed-d7f2f66dc548 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:05.852778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab11bd-4a2c-4216-99ed-d7f2f66dc548 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:05.853057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab11bd-4a2c-4216-99ed-d7f2f66dc548 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:05.853709 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab11bd-4a2c-4216-99ed-d7f2f66dc548 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:05.855152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab11bd-4a2c-4216-99ed-d7f2f66dc548 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:05.863390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:05.879263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 568/1138] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 21:02:05 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:02:08.071100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a539472-3298-4f0f-ac3f-da46fbea02f2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:08.071660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a539472-3298-4f0f-ac3f-da46fbea02f2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:08.071660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a539472-3298-4f0f-ac3f-da46fbea02f2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:08.072977 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 571/1139] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:02:08.079399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2522de-bd36-4bdd-8ffe-7ecc0931092f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:08.079813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2522de-bd36-4bdd-8ffe-7ecc0931092f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:08.080002 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2522de-bd36-4bdd-8ffe-7ecc0931092f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:08.102489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dd2522de-bd36-4bdd-8ffe-7ecc0931092f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:02:08.103675 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dd2522de-bd36-4bdd-8ffe-7ecc0931092f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:02:08.176585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 569/1140] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:02:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:02:08.447442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a539472-3298-4f0f-ac3f-da46fbea02f2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:08.517343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:08.563497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a539472-3298-4f0f-ac3f-da46fbea02f2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:08.600153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd22321-64c5-4e6b-92ed-abead3f33d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:08.600826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd22321-64c5-4e6b-92ed-abead3f33d2a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:08.601100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd22321-64c5-4e6b-92ed-abead3f33d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:08.601367 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd22321-64c5-4e6b-92ed-abead3f33d2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:08.612797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd22321-64c5-4e6b-92ed-abead3f33d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9bd22321-64c5-4e6b-92ed-abead3f33d2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:02:08.617594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:02:08.618652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:02:08.619691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:02:08.619691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:02:08.619691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:02:08.619691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:02:08.619691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:02:08.619691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:02:08.619691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:02:08.619691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:02:08.619691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:02:08.619691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:02:08.619691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:02:08.619691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:02:08.619691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 572/1141] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:02:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:02:08.625658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2522de-bd36-4bdd-8ffe-7ecc0931092f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:08.736605 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2522de-bd36-4bdd-8ffe-7ecc0931092f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:08.758288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:08.774378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7451715-3c64-4653-bc17-c85725cfa228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:08.774969 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7451715-3c64-4653-bc17-c85725cfa228 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:08.775244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7451715-3c64-4653-bc17-c85725cfa228 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:08.775507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7451715-3c64-4653-bc17-c85725cfa228 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:08.776162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7451715-3c64-4653-bc17-c85725cfa228 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:02:08.776454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7451715-3c64-4653-bc17-c85725cfa228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:08.777855 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7451715-3c64-4653-bc17-c85725cfa228 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:08.785827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 570/1142] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:02:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:08.792202 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd22321-64c5-4e6b-92ed-abead3f33d2a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:08.909854 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd22321-64c5-4e6b-92ed-abead3f33d2a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:08.929640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:08.952874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bd616-b813-4dcb-b75e-6440f4d9a0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:08.953497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bd616-b813-4dcb-b75e-6440f4d9a0ea None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:08.953774 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bd616-b813-4dcb-b75e-6440f4d9a0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:08.954042 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bd616-b813-4dcb-b75e-6440f4d9a0ea None admin] PATH_INFO: `/v3/users/c49139dbdcf24f66ad3d4f3dc9283372` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:08.969116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bd616-b813-4dcb-b75e-6440f4d9a0ea None admin] RBAC: Authorizing `identity:get_user(user_id=c49139dbdcf24f66ad3d4f3dc9283372)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:08.970622 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bd616-b813-4dcb-b75e-6440f4d9a0ea None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:08.972250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 573/1143] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:02:08 2022] GET /identity/v3/users/c49139dbdcf24f66ad3d4f3dc9283372 => generated 286 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:08.978432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7451715-3c64-4653-bc17-c85725cfa228 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:09.084755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7451715-3c64-4653-bc17-c85725cfa228 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:09.119396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ace51f-5aad-47df-9664-682c4a04e107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:09.119992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ace51f-5aad-47df-9664-682c4a04e107 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:09.120265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ace51f-5aad-47df-9664-682c4a04e107 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:09.120525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ace51f-5aad-47df-9664-682c4a04e107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:09.172218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:09.194493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ace51f-5aad-47df-9664-682c4a04e107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:09.195977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ace51f-5aad-47df-9664-682c4a04e107 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:09.295902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 571/1144] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:02:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:02:09.302529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bd616-b813-4dcb-b75e-6440f4d9a0ea None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:09.407431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bd616-b813-4dcb-b75e-6440f4d9a0ea None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:09.443656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c285a84-faf1-475f-9cbc-a6a301d7c762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:09.444279 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c285a84-faf1-475f-9cbc-a6a301d7c762 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:09.444560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c285a84-faf1-475f-9cbc-a6a301d7c762 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:09.444864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c285a84-faf1-475f-9cbc-a6a301d7c762 None admin] PATH_INFO: `/v3/projects/2f81b147e1144cb2be4720e7bf35c1ea` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:09.446094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c285a84-faf1-475f-9cbc-a6a301d7c762 None admin] RBAC: Authorizing `identity:get_project(project_id=2f81b147e1144cb2be4720e7bf35c1ea)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:09.447543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c285a84-faf1-475f-9cbc-a6a301d7c762 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:09.449212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 574/1145] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:02:09 2022] GET /identity/v3/projects/2f81b147e1144cb2be4720e7bf35c1ea => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:09.456518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ace51f-5aad-47df-9664-682c4a04e107 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:09.548958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:09.582151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ace51f-5aad-47df-9664-682c4a04e107 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:09.621786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a2ce3-57b8-448a-ab23-6d7d53481136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:09.622369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a2ce3-57b8-448a-ab23-6d7d53481136 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:09.622684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a2ce3-57b8-448a-ab23-6d7d53481136 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:09.622946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a2ce3-57b8-448a-ab23-6d7d53481136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:09.623691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a2ce3-57b8-448a-ab23-6d7d53481136 None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=2f81b147e1144cb2be4720e7bf35c1ea, user.id=c49139dbdcf24f66ad3d4f3dc9283372) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:02:09.624019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a2ce3-57b8-448a-ab23-6d7d53481136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:09.625492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a2ce3-57b8-448a-ab23-6d7d53481136 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:09.630968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 572/1146] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:02:09 2022] GET /identity/v3/role_assignments?user.id=c49139dbdcf24f66ad3d4f3dc9283372&scope.project.id=2f81b147e1144cb2be4720e7bf35c1ea&role.id=3b30504991194002b41f8587d0851c10 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:11.768330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5a717-c7b7-4094-b2f9-64b87e5eb6ca None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:11.768970 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5a717-c7b7-4094-b2f9-64b87e5eb6ca None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:11.768970 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5a717-c7b7-4094-b2f9-64b87e5eb6ca None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:11.770275 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 575/1147] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:02:11.776584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1408d-13b0-4611-9b4b-4d55af3d9d0e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:11.777346 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1408d-13b0-4611-9b4b-4d55af3d9d0e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:11.777488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1408d-13b0-4611-9b4b-4d55af3d9d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:11.803766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-58e1408d-13b0-4611-9b4b-4d55af3d9d0e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:02:11.804992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-58e1408d-13b0-4611-9b4b-4d55af3d9d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:02:11.876360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 573/1148] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:02:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:02:12.150553 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5a717-c7b7-4094-b2f9-64b87e5eb6ca None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:12.204443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:12.263987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5a717-c7b7-4094-b2f9-64b87e5eb6ca None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:12.300329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b27c1-f724-4e6c-8b0d-cfa221eedbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:12.300974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b27c1-f724-4e6c-8b0d-cfa221eedbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:12.301249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b27c1-f724-4e6c-8b0d-cfa221eedbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:12.301515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b27c1-f724-4e6c-8b0d-cfa221eedbb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:12.313042 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b27c1-f724-4e6c-8b0d-cfa221eedbb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c54b27c1-f724-4e6c-8b0d-cfa221eedbb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:02:12.317791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:02:12.319112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:02:12.320582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:02:12.320582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:02:12.320582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:02:12.320582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:02:12.320582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:02:12.320582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:02:12.320582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:02:12.320582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:02:12.320582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:02:12.320582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:02:12.320582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:02:12.320582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:02:12.320582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 576/1149] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:02:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:02:12.328446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1408d-13b0-4611-9b4b-4d55af3d9d0e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:12.410215 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:12.440132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1408d-13b0-4611-9b4b-4d55af3d9d0e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:12.480583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c52c1-11dd-4f7f-b632-605a21952fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:12.481220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c52c1-11dd-4f7f-b632-605a21952fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:12.481498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c52c1-11dd-4f7f-b632-605a21952fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:12.481767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c52c1-11dd-4f7f-b632-605a21952fd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:12.482441 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c52c1-11dd-4f7f-b632-605a21952fd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:02:12.482745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c52c1-11dd-4f7f-b632-605a21952fd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:12.484172 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c52c1-11dd-4f7f-b632-605a21952fd5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:12.492164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 574/1150] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:02:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:12.498988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b27c1-f724-4e6c-8b0d-cfa221eedbb1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:12.608868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b27c1-f724-4e6c-8b0d-cfa221eedbb1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:12.611360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:12.644370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f7f9e-51aa-4394-b547-d949a2bc72e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:12.645021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f7f9e-51aa-4394-b547-d949a2bc72e2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:12.645296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f7f9e-51aa-4394-b547-d949a2bc72e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:12.645565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f7f9e-51aa-4394-b547-d949a2bc72e2 None admin] PATH_INFO: `/v3/users/c49139dbdcf24f66ad3d4f3dc9283372` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:12.646692 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f7f9e-51aa-4394-b547-d949a2bc72e2 None admin] RBAC: Authorizing `identity:get_user(user_id=c49139dbdcf24f66ad3d4f3dc9283372)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:12.648132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f7f9e-51aa-4394-b547-d949a2bc72e2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:12.649734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 577/1151] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:02:12 2022] GET /identity/v3/users/c49139dbdcf24f66ad3d4f3dc9283372 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:12.655976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c52c1-11dd-4f7f-b632-605a21952fd5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:12.760199 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c52c1-11dd-4f7f-b632-605a21952fd5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:12.795769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4740f-2555-42c4-8ed0-78e233f953f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:12.796387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4740f-2555-42c4-8ed0-78e233f953f5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:12.796678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4740f-2555-42c4-8ed0-78e233f953f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:12.797015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4740f-2555-42c4-8ed0-78e233f953f5 None admin] PATH_INFO: `/v3/projects/2f81b147e1144cb2be4720e7bf35c1ea` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:12.798309 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4740f-2555-42c4-8ed0-78e233f953f5 None admin] RBAC: Authorizing `identity:get_project(project_id=2f81b147e1144cb2be4720e7bf35c1ea)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:12.799754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4740f-2555-42c4-8ed0-78e233f953f5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:12.801434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 575/1152] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:02:12 2022] GET /identity/v3/projects/2f81b147e1144cb2be4720e7bf35c1ea => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:02:12.808300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f7f9e-51aa-4394-b547-d949a2bc72e2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:12.912149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f7f9e-51aa-4394-b547-d949a2bc72e2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:12.948128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2950328-4abd-476b-b75f-031671274b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:12.948798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2950328-4abd-476b-b75f-031671274b8b None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:12.949078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2950328-4abd-476b-b75f-031671274b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:12.949344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2950328-4abd-476b-b75f-031671274b8b None admin] PATH_INFO: `/v3/projects/2f81b147e1144cb2be4720e7bf35c1ea/users/c49139dbdcf24f66ad3d4f3dc9283372/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:12.951488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2950328-4abd-476b-b75f-031671274b8b None admin] RBAC: Authorizing `identity:create_grant(project_id=2f81b147e1144cb2be4720e7bf35c1ea, user_id=c49139dbdcf24f66ad3d4f3dc9283372, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:12.953021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2950328-4abd-476b-b75f-031671274b8b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:12.973094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 578/1153] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:02:12 2022] PUT /identity/v3/projects/2f81b147e1144cb2be4720e7bf35c1ea/users/c49139dbdcf24f66ad3d4f3dc9283372/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:02:15.089405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ae5e2-f3dc-485c-b9ed-d7ea4a1f373e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:15.090298 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ae5e2-f3dc-485c-b9ed-d7ea4a1f373e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:15.090298 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ae5e2-f3dc-485c-b9ed-d7ea4a1f373e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:15.092570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 576/1154] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:02:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:02:15.098118 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac584ad-c3c6-49c7-aed6-b3c15acd7eba None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:15.098561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac584ad-c3c6-49c7-aed6-b3c15acd7eba None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:15.098561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac584ad-c3c6-49c7-aed6-b3c15acd7eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:15.117539 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fac584ad-c3c6-49c7-aed6-b3c15acd7eba None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:02:15.118721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fac584ad-c3c6-49c7-aed6-b3c15acd7eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:02:15.146388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:15.196140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 579/1155] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:02:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:02:15.466707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ae5e2-f3dc-485c-b9ed-d7ea4a1f373e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:15.482627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:15.596347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ae5e2-f3dc-485c-b9ed-d7ea4a1f373e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:15.632343 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ecde6-806f-4fe2-97d4-d76480fcf407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:15.632993 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ecde6-806f-4fe2-97d4-d76480fcf407 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:15.633272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ecde6-806f-4fe2-97d4-d76480fcf407 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:15.633539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ecde6-806f-4fe2-97d4-d76480fcf407 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:15.645155 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ecde6-806f-4fe2-97d4-d76480fcf407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-328ecde6-806f-4fe2-97d4-d76480fcf407 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:02:15.650028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:02:15.651108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:02:15.652405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:02:15.652405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:02:15.652405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:02:15.652405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:02:15.652405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:02:15.652405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:02:15.652405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:02:15.652405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:02:15.652405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:02:15.652405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:02:15.652405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:02:15.652405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:02:15.652405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 577/1156] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:02:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:02:15.657651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac584ad-c3c6-49c7-aed6-b3c15acd7eba None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:15.765446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac584ad-c3c6-49c7-aed6-b3c15acd7eba None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:15.800965 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec3dba-3d56-4748-9793-a31c95b4e7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:15.801669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec3dba-3d56-4748-9793-a31c95b4e7bc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:15.801957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec3dba-3d56-4748-9793-a31c95b4e7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:15.802208 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec3dba-3d56-4748-9793-a31c95b4e7bc None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:15.802874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec3dba-3d56-4748-9793-a31c95b4e7bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:02:15.803173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec3dba-3d56-4748-9793-a31c95b4e7bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:15.804564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec3dba-3d56-4748-9793-a31c95b4e7bc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:15.813317 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 580/1157] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:02:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:15.820071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ecde6-806f-4fe2-97d4-d76480fcf407 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:15.902637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:15.951293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ecde6-806f-4fe2-97d4-d76480fcf407 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:15.995828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936f6a-10b2-480a-a060-f06cb003f67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:15.996817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936f6a-10b2-480a-a060-f06cb003f67f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:15.997206 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936f6a-10b2-480a-a060-f06cb003f67f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:15.997585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936f6a-10b2-480a-a060-f06cb003f67f None admin] PATH_INFO: `/v3/users/c49139dbdcf24f66ad3d4f3dc9283372` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:15.999606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e936f6a-10b2-480a-a060-f06cb003f67f None admin] RBAC: Authorizing `identity:get_user(user_id=c49139dbdcf24f66ad3d4f3dc9283372)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:16.001957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e936f6a-10b2-480a-a060-f06cb003f67f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:16.004492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 578/1158] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:02:15 2022] GET /identity/v3/users/c49139dbdcf24f66ad3d4f3dc9283372 => generated 286 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:16.013002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec3dba-3d56-4748-9793-a31c95b4e7bc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:16.041006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:16.121329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec3dba-3d56-4748-9793-a31c95b4e7bc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:16.156518 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4794095-ae26-495e-9157-4a3720ded68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:16.157219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4794095-ae26-495e-9157-4a3720ded68e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:16.157503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4794095-ae26-495e-9157-4a3720ded68e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:16.157771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4794095-ae26-495e-9157-4a3720ded68e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:16.232494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4794095-ae26-495e-9157-4a3720ded68e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:16.234152 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4794095-ae26-495e-9157-4a3720ded68e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:16.262054 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:16.347991 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 581/1159] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:02:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:02:16.354417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936f6a-10b2-480a-a060-f06cb003f67f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:16.470463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936f6a-10b2-480a-a060-f06cb003f67f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:16.508441 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ee538-a914-45fa-b588-86a3fdc74047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:16.509092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ee538-a914-45fa-b588-86a3fdc74047 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:16.509367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ee538-a914-45fa-b588-86a3fdc74047 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:16.509642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ee538-a914-45fa-b588-86a3fdc74047 None admin] PATH_INFO: `/v3/projects/2f81b147e1144cb2be4720e7bf35c1ea` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:16.510895 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ee538-a914-45fa-b588-86a3fdc74047 None admin] RBAC: Authorizing `identity:get_project(project_id=2f81b147e1144cb2be4720e7bf35c1ea)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:16.512341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ee538-a914-45fa-b588-86a3fdc74047 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:16.514030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 579/1160] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:02:16 2022] GET /identity/v3/projects/2f81b147e1144cb2be4720e7bf35c1ea => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:16.524276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4794095-ae26-495e-9157-4a3720ded68e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:16.638559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4794095-ae26-495e-9157-4a3720ded68e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:16.662328 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:16.675372 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21425c9-8a4c-46a3-b013-36b95cd2f7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:16.675976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21425c9-8a4c-46a3-b013-36b95cd2f7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:16.676250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21425c9-8a4c-46a3-b013-36b95cd2f7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:16.676518 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21425c9-8a4c-46a3-b013-36b95cd2f7a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:16.677271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21425c9-8a4c-46a3-b013-36b95cd2f7a0 None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=2f81b147e1144cb2be4720e7bf35c1ea, user.id=c49139dbdcf24f66ad3d4f3dc9283372) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:02:16.677576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21425c9-8a4c-46a3-b013-36b95cd2f7a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:16.679044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21425c9-8a4c-46a3-b013-36b95cd2f7a0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:16.685316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 582/1161] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:02:16 2022] GET /identity/v3/role_assignments?user.id=c49139dbdcf24f66ad3d4f3dc9283372&scope.project.id=2f81b147e1144cb2be4720e7bf35c1ea&role.id=3b30504991194002b41f8587d0851c10 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:18.856549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d8dd6-6d71-42b4-87dd-daf47fe2bed7 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:18.857423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d8dd6-6d71-42b4-87dd-daf47fe2bed7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:18.857423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d8dd6-6d71-42b4-87dd-daf47fe2bed7 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:18.859532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 580/1162] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:02:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:02:18.865947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb52c1a-d374-44ea-a1fe-ba717a8eb9f5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:18.866412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb52c1a-d374-44ea-a1fe-ba717a8eb9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:18.866504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb52c1a-d374-44ea-a1fe-ba717a8eb9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:18.889619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bcb52c1a-d374-44ea-a1fe-ba717a8eb9f5 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:02:18.890858 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bcb52c1a-d374-44ea-a1fe-ba717a8eb9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:02:18.963381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 583/1163] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:02:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:02:19.234581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d8dd6-6d71-42b4-87dd-daf47fe2bed7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:19.239025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:19.368014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d8dd6-6d71-42b4-87dd-daf47fe2bed7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:19.404057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac5a2a-f3c2-4ad1-b107-9c80d3b4830a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:19.404691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac5a2a-f3c2-4ad1-b107-9c80d3b4830a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:19.404970 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac5a2a-f3c2-4ad1-b107-9c80d3b4830a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:19.405240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac5a2a-f3c2-4ad1-b107-9c80d3b4830a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:19.486192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac5a2a-f3c2-4ad1-b107-9c80d3b4830a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:19.487562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac5a2a-f3c2-4ad1-b107-9c80d3b4830a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:19.489516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:19.598834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 581/1164] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:02:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:02:19.608225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb52c1a-d374-44ea-a1fe-ba717a8eb9f5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:19.721412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb52c1a-d374-44ea-a1fe-ba717a8eb9f5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:19.738774 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:19.767049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00278ef8-176e-4fd3-9d18-f73e4920339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:19.767681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00278ef8-176e-4fd3-9d18-f73e4920339c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:19.767952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00278ef8-176e-4fd3-9d18-f73e4920339c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:19.768217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00278ef8-176e-4fd3-9d18-f73e4920339c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:19.774846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00278ef8-176e-4fd3-9d18-f73e4920339c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:19.776296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00278ef8-176e-4fd3-9d18-f73e4920339c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-00278ef8-176e-4fd3-9d18-f73e4920339c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:02:19.787575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:02:19.788734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:02:19.789614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 584/1165] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Sep 6 21:02:19 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:02:19.795137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac5a2a-f3c2-4ad1-b107-9c80d3b4830a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:19.899219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac5a2a-f3c2-4ad1-b107-9c80d3b4830a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:19.901693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:19.936495 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9352cf-916f-4e06-9341-70a623ae265c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:19.937107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9352cf-916f-4e06-9341-70a623ae265c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:19.937377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9352cf-916f-4e06-9341-70a623ae265c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:19.937640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9352cf-916f-4e06-9341-70a623ae265c None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:19.938303 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9352cf-916f-4e06-9341-70a623ae265c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:02:19.938597 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9352cf-916f-4e06-9341-70a623ae265c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:19.939960 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9352cf-916f-4e06-9341-70a623ae265c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:19.947454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 582/1166] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Sep 6 21:02:19 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:22.073295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912bbe7-75cc-47e5-bff5-60d76affff16 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:22.074187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912bbe7-75cc-47e5-bff5-60d76affff16 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:22.074187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912bbe7-75cc-47e5-bff5-60d76affff16 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:22.076303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 585/1167] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:02:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:02:22.083136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6cb13-563b-4d06-b30e-113b361a4fc1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:22.083550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6cb13-563b-4d06-b30e-113b361a4fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:22.083681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6cb13-563b-4d06-b30e-113b361a4fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:22.103756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-67f6cb13-563b-4d06-b30e-113b361a4fc1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:02:22.104994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-67f6cb13-563b-4d06-b30e-113b361a4fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:02:22.180092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 583/1168] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:02:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:02:22.446756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912bbe7-75cc-47e5-bff5-60d76affff16 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:22.570839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912bbe7-75cc-47e5-bff5-60d76affff16 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:22.606625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f55c8-25c9-4ab5-aba8-0424e157d51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:22.607247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f55c8-25c9-4ab5-aba8-0424e157d51f None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:22.607528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f55c8-25c9-4ab5-aba8-0424e157d51f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:22.607798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f55c8-25c9-4ab5-aba8-0424e157d51f None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:22.608474 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f55c8-25c9-4ab5-aba8-0424e157d51f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:22.609914 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f55c8-25c9-4ab5-aba8-0424e157d51f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:22.628112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 586/1169] 10.222.0.79 () {60 vars in 1138 bytes} [Tue Sep 6 21:02:22 2022] POST /identity/v3/domains => generated 267 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:02:24.825624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3982e-50da-4b62-95e3-4a79446570cd None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:24.826187 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3982e-50da-4b62-95e3-4a79446570cd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:24.826187 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3982e-50da-4b62-95e3-4a79446570cd None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:24.827529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 584/1170] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:02:24.834085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b96cd-6afa-4f64-b07d-4450aeeb0162 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:24.834797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b96cd-6afa-4f64-b07d-4450aeeb0162 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:24.834920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b96cd-6afa-4f64-b07d-4450aeeb0162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:24.857838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f52b96cd-6afa-4f64-b07d-4450aeeb0162 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:02:24.858961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f52b96cd-6afa-4f64-b07d-4450aeeb0162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:02:24.889965 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:24.935762 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 587/1171] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:02:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:02:25.206067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3982e-50da-4b62-95e3-4a79446570cd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:25.316926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:25.328132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3982e-50da-4b62-95e3-4a79446570cd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:25.372138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69c696-67cc-4959-9628-16c2ecce24eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:25.372795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69c696-67cc-4959-9628-16c2ecce24eb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:25.373079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69c696-67cc-4959-9628-16c2ecce24eb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:25.373349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69c696-67cc-4959-9628-16c2ecce24eb None admin] PATH_INFO: `/v3/domains/52a6b7a1840c415385707e6edfdd2ef6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:25.384217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69c696-67cc-4959-9628-16c2ecce24eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=52a6b7a1840c415385707e6edfdd2ef6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:25.385700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69c696-67cc-4959-9628-16c2ecce24eb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:25.387312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 585/1172] 10.222.0.79 () {56 vars in 1183 bytes} [Tue Sep 6 21:02:25 2022] GET /identity/v3/domains/52a6b7a1840c415385707e6edfdd2ef6 => generated 267 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:25.393967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b96cd-6afa-4f64-b07d-4450aeeb0162 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:25.503632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b96cd-6afa-4f64-b07d-4450aeeb0162 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:25.547128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357ef4c-f212-40d0-b332-68c3bc160948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:25.547761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357ef4c-f212-40d0-b332-68c3bc160948 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:25.548035 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357ef4c-f212-40d0-b332-68c3bc160948 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:25.548300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357ef4c-f212-40d0-b332-68c3bc160948 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:25.548995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357ef4c-f212-40d0-b332-68c3bc160948 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:25.550453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357ef4c-f212-40d0-b332-68c3bc160948 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:25.581102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 588/1173] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 21:02:25 2022] POST /identity/v3/projects => generated 361 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:02:27.753538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c117b4f-1ec1-40df-986a-cd647c16af8b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:27.754397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c117b4f-1ec1-40df-986a-cd647c16af8b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:27.754397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c117b4f-1ec1-40df-986a-cd647c16af8b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:27.756809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 586/1174] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:02:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:02:27.763033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216738f3-01d1-43df-8415-e1ad6e5e666d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:27.763784 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216738f3-01d1-43df-8415-e1ad6e5e666d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:27.763906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216738f3-01d1-43df-8415-e1ad6e5e666d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:27.790247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-216738f3-01d1-43df-8415-e1ad6e5e666d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:02:27.791474 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-216738f3-01d1-43df-8415-e1ad6e5e666d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:02:27.806721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:27.862617 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 589/1175] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:02:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:02:28.131649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c117b4f-1ec1-40df-986a-cd647c16af8b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:28.250569 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c117b4f-1ec1-40df-986a-cd647c16af8b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:28.285689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22021fdc-f347-4dcd-8b62-ecf95c0711ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:28.286314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021fdc-f347-4dcd-8b62-ecf95c0711ce None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:28.286593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021fdc-f347-4dcd-8b62-ecf95c0711ce None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:28.286861 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021fdc-f347-4dcd-8b62-ecf95c0711ce None admin] PATH_INFO: `/v3/domains/52a6b7a1840c415385707e6edfdd2ef6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:28.288078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22021fdc-f347-4dcd-8b62-ecf95c0711ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=52a6b7a1840c415385707e6edfdd2ef6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:28.289630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22021fdc-f347-4dcd-8b62-ecf95c0711ce None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:28.292362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 587/1176] 10.222.0.79 () {56 vars in 1183 bytes} [Tue Sep 6 21:02:28 2022] GET /identity/v3/domains/52a6b7a1840c415385707e6edfdd2ef6 => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:28.299390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216738f3-01d1-43df-8415-e1ad6e5e666d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:28.422122 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216738f3-01d1-43df-8415-e1ad6e5e666d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:28.457666 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f66b218-5ccf-452a-8d72-5497b9932206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:28.458251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66b218-5ccf-452a-8d72-5497b9932206 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:28.458526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66b218-5ccf-452a-8d72-5497b9932206 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:28.458792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66b218-5ccf-452a-8d72-5497b9932206 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:28.459458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f66b218-5ccf-452a-8d72-5497b9932206 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:28.460962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f66b218-5ccf-452a-8d72-5497b9932206 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:28.487554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 590/1177] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 21:02:28 2022] POST /identity/v3/users => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:02:30.680024 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00982-0f1d-486e-9361-e52699c13334 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:30.680608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00982-0f1d-486e-9361-e52699c13334 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:30.680608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00982-0f1d-486e-9361-e52699c13334 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:30.682072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 588/1178] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:02:30.689118 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a0b3a-b801-4c71-9b49-d790c735bb98 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:30.689876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a0b3a-b801-4c71-9b49-d790c735bb98 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:30.689994 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a0b3a-b801-4c71-9b49-d790c735bb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:30.712095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:30.720940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5d1a0b3a-b801-4c71-9b49-d790c735bb98 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:02:30.722684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5d1a0b3a-b801-4c71-9b49-d790c735bb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:02:30.796681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 591/1179] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:02:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:02:31.082347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e00982-0f1d-486e-9361-e52699c13334 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:31.121218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:31.222790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e00982-0f1d-486e-9361-e52699c13334 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:31.261298 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d38337-ee03-4e8b-92fe-2b494263f1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:31.261945 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d38337-ee03-4e8b-92fe-2b494263f1cf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:31.262223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d38337-ee03-4e8b-92fe-2b494263f1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:31.262491 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d38337-ee03-4e8b-92fe-2b494263f1cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:31.273337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d38337-ee03-4e8b-92fe-2b494263f1cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-78d38337-ee03-4e8b-92fe-2b494263f1cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:02:31.278194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:02:31.279212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:02:31.280175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:02:31.280175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:02:31.280175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:02:31.280175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:02:31.280175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:02:31.280175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:02:31.280175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:02:31.280175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:02:31.280175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:02:31.280175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:02:31.280175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:02:31.280175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:02:31.280175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 589/1180] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:02:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:02:31.287555 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a0b3a-b801-4c71-9b49-d790c735bb98 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:31.398420 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a0b3a-b801-4c71-9b49-d790c735bb98 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:31.434476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8ae01-c18f-445e-885f-af2aa5c9101b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:31.435429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8ae01-c18f-445e-885f-af2aa5c9101b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:31.435697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8ae01-c18f-445e-885f-af2aa5c9101b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:31.435962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8ae01-c18f-445e-885f-af2aa5c9101b None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:31.436623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8ae01-c18f-445e-885f-af2aa5c9101b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:02:31.436947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8ae01-c18f-445e-885f-af2aa5c9101b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:31.438404 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8ae01-c18f-445e-885f-af2aa5c9101b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:31.440892 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:31.447950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 592/1181] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:02:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:31.454437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d38337-ee03-4e8b-92fe-2b494263f1cf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:31.542824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:31.560110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d38337-ee03-4e8b-92fe-2b494263f1cf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:31.603515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a57ad-f3c7-4143-8468-bc404b224df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:31.604438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a57ad-f3c7-4143-8468-bc404b224df6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:31.604872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a57ad-f3c7-4143-8468-bc404b224df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:31.605261 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a57ad-f3c7-4143-8468-bc404b224df6 None admin] PATH_INFO: `/v3/users/63566e537ba948fdbae7fd83529539c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:31.623810 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a57ad-f3c7-4143-8468-bc404b224df6 None admin] RBAC: Authorizing `identity:get_user(user_id=63566e537ba948fdbae7fd83529539c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:31.626054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a57ad-f3c7-4143-8468-bc404b224df6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:31.628048 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 590/1182] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:02:31 2022] GET /identity/v3/users/63566e537ba948fdbae7fd83529539c5 => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:31.634809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8ae01-c18f-445e-885f-af2aa5c9101b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:31.750875 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8ae01-c18f-445e-885f-af2aa5c9101b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:31.786295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e9c7f-79bc-4bea-a84f-19c461b978a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:31.786896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9c7f-79bc-4bea-a84f-19c461b978a8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:31.787169 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9c7f-79bc-4bea-a84f-19c461b978a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:31.787437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9c7f-79bc-4bea-a84f-19c461b978a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:31.854018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:31.858645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e9c7f-79bc-4bea-a84f-19c461b978a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:31.860289 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e9c7f-79bc-4bea-a84f-19c461b978a8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:31.978678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 593/1183] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:02:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:02:31.985290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a57ad-f3c7-4143-8468-bc404b224df6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:32.095782 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a57ad-f3c7-4143-8468-bc404b224df6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:32.131201 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f29a25-9d17-479e-9b81-106fcff9663d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:32.131818 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29a25-9d17-479e-9b81-106fcff9663d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:32.132095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29a25-9d17-479e-9b81-106fcff9663d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:32.132368 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29a25-9d17-479e-9b81-106fcff9663d None admin] PATH_INFO: `/v3/projects/4e2b0fab14d44cb89a8b4eacfeb4f818` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:32.133572 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f29a25-9d17-479e-9b81-106fcff9663d None admin] RBAC: Authorizing `identity:get_project(project_id=4e2b0fab14d44cb89a8b4eacfeb4f818)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:32.135020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f29a25-9d17-479e-9b81-106fcff9663d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:32.136675 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 591/1184] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:02:31 2022] GET /identity/v3/projects/4e2b0fab14d44cb89a8b4eacfeb4f818 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:32.143818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e9c7f-79bc-4bea-a84f-19c461b978a8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:32.252983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:32.257106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e9c7f-79bc-4bea-a84f-19c461b978a8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:32.293748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106affa4-cd64-4bdc-bc19-aae9063c5994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:32.294352 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106affa4-cd64-4bdc-bc19-aae9063c5994 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:32.294631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106affa4-cd64-4bdc-bc19-aae9063c5994 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:32.294898 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106affa4-cd64-4bdc-bc19-aae9063c5994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:32.295618 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106affa4-cd64-4bdc-bc19-aae9063c5994 None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=4e2b0fab14d44cb89a8b4eacfeb4f818, user.id=63566e537ba948fdbae7fd83529539c5) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:02:32.295926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106affa4-cd64-4bdc-bc19-aae9063c5994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:32.297432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106affa4-cd64-4bdc-bc19-aae9063c5994 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:32.302917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 594/1185] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:02:32 2022] GET /identity/v3/role_assignments?user.id=63566e537ba948fdbae7fd83529539c5&scope.project.id=4e2b0fab14d44cb89a8b4eacfeb4f818&role.id=3b30504991194002b41f8587d0851c10 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:34.423134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2b641-3c66-4d6f-8f8e-4c8b47bb5205 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:34.423793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2b641-3c66-4d6f-8f8e-4c8b47bb5205 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:34.423793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2b641-3c66-4d6f-8f8e-4c8b47bb5205 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:34.425068 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 592/1186] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:02:34.431504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96de895-cb98-4901-9096-249eff52d907 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:34.432241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96de895-cb98-4901-9096-249eff52d907 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:34.432412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96de895-cb98-4901-9096-249eff52d907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:34.455375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b96de895-cb98-4901-9096-249eff52d907 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:02:34.456583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b96de895-cb98-4901-9096-249eff52d907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:02:34.531113 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 595/1187] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:02:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:02:34.816832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2b641-3c66-4d6f-8f8e-4c8b47bb5205 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:34.836515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:34.934430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2b641-3c66-4d6f-8f8e-4c8b47bb5205 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:34.971805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a67bf-19c9-4f4d-959a-e72777ef4072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:34.972421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a67bf-19c9-4f4d-959a-e72777ef4072 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:34.972700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a67bf-19c9-4f4d-959a-e72777ef4072 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:34.973044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a67bf-19c9-4f4d-959a-e72777ef4072 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:34.983231 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a67bf-19c9-4f4d-959a-e72777ef4072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-539a67bf-19c9-4f4d-959a-e72777ef4072 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:02:34.992103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:02:34.993389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:02:34.994422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:02:34.994422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:02:34.994422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:02:34.994422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:02:34.994422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:02:34.994422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:02:34.994422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:02:34.994422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:02:34.994422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:02:34.994422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:02:34.994422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:02:34.994422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:02:34.995656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 593/1188] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:02:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:02:35.004031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96de895-cb98-4901-9096-249eff52d907 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:35.127440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:35.138156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96de895-cb98-4901-9096-249eff52d907 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:35.182084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6d520-842c-4888-aa93-724fbcf53e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:35.182701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6d520-842c-4888-aa93-724fbcf53e47 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:35.182977 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6d520-842c-4888-aa93-724fbcf53e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:35.183251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6d520-842c-4888-aa93-724fbcf53e47 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:35.183916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6d520-842c-4888-aa93-724fbcf53e47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:02:35.184215 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6d520-842c-4888-aa93-724fbcf53e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:35.185979 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6d520-842c-4888-aa93-724fbcf53e47 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:35.194408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 596/1189] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:02:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:35.201078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a67bf-19c9-4f4d-959a-e72777ef4072 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:35.277197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:35.324975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a67bf-19c9-4f4d-959a-e72777ef4072 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:35.360938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60536c2-9e6f-4e38-a39f-9a299ce90e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:35.361564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60536c2-9e6f-4e38-a39f-9a299ce90e15 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:35.361808 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60536c2-9e6f-4e38-a39f-9a299ce90e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:35.362080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60536c2-9e6f-4e38-a39f-9a299ce90e15 None admin] PATH_INFO: `/v3/users/63566e537ba948fdbae7fd83529539c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:35.363278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60536c2-9e6f-4e38-a39f-9a299ce90e15 None admin] RBAC: Authorizing `identity:get_user(user_id=63566e537ba948fdbae7fd83529539c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:35.364765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60536c2-9e6f-4e38-a39f-9a299ce90e15 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:35.366331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 594/1190] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:02:35 2022] GET /identity/v3/users/63566e537ba948fdbae7fd83529539c5 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:35.376024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6d520-842c-4888-aa93-724fbcf53e47 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:35.482701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6d520-842c-4888-aa93-724fbcf53e47 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:35.520965 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d76f69-dab3-4780-b073-e86faa991567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:35.521596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d76f69-dab3-4780-b073-e86faa991567 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:35.521873 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d76f69-dab3-4780-b073-e86faa991567 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:35.522142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d76f69-dab3-4780-b073-e86faa991567 None admin] PATH_INFO: `/v3/projects/4e2b0fab14d44cb89a8b4eacfeb4f818` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:35.523483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d76f69-dab3-4780-b073-e86faa991567 None admin] RBAC: Authorizing `identity:get_project(project_id=4e2b0fab14d44cb89a8b4eacfeb4f818)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:35.525014 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d76f69-dab3-4780-b073-e86faa991567 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:35.526699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 597/1191] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:02:35 2022] GET /identity/v3/projects/4e2b0fab14d44cb89a8b4eacfeb4f818 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:35.533341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60536c2-9e6f-4e38-a39f-9a299ce90e15 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:35.647964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60536c2-9e6f-4e38-a39f-9a299ce90e15 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:35.684352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cce58-8249-4913-a7c9-0fe3c283fd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:35.685392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cce58-8249-4913-a7c9-0fe3c283fd79 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:35.685665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cce58-8249-4913-a7c9-0fe3c283fd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:35.685928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cce58-8249-4913-a7c9-0fe3c283fd79 None admin] PATH_INFO: `/v3/projects/4e2b0fab14d44cb89a8b4eacfeb4f818/users/63566e537ba948fdbae7fd83529539c5/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:35.688141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cce58-8249-4913-a7c9-0fe3c283fd79 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e2b0fab14d44cb89a8b4eacfeb4f818, user_id=63566e537ba948fdbae7fd83529539c5, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:35.689628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cce58-8249-4913-a7c9-0fe3c283fd79 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:35.701893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:35.708458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 595/1192] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:02:35 2022] PUT /identity/v3/projects/4e2b0fab14d44cb89a8b4eacfeb4f818/users/63566e537ba948fdbae7fd83529539c5/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:02:37.842857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f68aa-5ab1-4507-ac12-7944c1e85f94 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:37.843673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f68aa-5ab1-4507-ac12-7944c1e85f94 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:37.843673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f68aa-5ab1-4507-ac12-7944c1e85f94 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:37.845844 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 598/1193] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:02:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:02:37.852173 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ded5f8-d1f0-47b7-bac8-233b57c17852 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:37.852899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ded5f8-d1f0-47b7-bac8-233b57c17852 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:37.853073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ded5f8-d1f0-47b7-bac8-233b57c17852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:37.875411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-24ded5f8-d1f0-47b7-bac8-233b57c17852 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:02:37.876610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-24ded5f8-d1f0-47b7-bac8-233b57c17852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:02:37.949944 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 596/1194] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:02:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:02:38.216458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f68aa-5ab1-4507-ac12-7944c1e85f94 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:38.245265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:38.328114 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f68aa-5ab1-4507-ac12-7944c1e85f94 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:38.377162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731c5b8-1c5e-4345-8447-1a0cb1a1ea86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:38.377790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731c5b8-1c5e-4345-8447-1a0cb1a1ea86 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:38.378069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731c5b8-1c5e-4345-8447-1a0cb1a1ea86 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:38.378340 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731c5b8-1c5e-4345-8447-1a0cb1a1ea86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:38.389071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731c5b8-1c5e-4345-8447-1a0cb1a1ea86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8731c5b8-1c5e-4345-8447-1a0cb1a1ea86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:02:38.393939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:02:38.395043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:02:38.396737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:02:38.396737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:02:38.396737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:02:38.396737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:02:38.396737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:02:38.396737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:02:38.396737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:02:38.396737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:02:38.396737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:02:38.396737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:02:38.396737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:02:38.396737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:02:38.396737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 599/1195] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:02:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:02:38.404952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ded5f8-d1f0-47b7-bac8-233b57c17852 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:38.508419 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ded5f8-d1f0-47b7-bac8-233b57c17852 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:38.548333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:38.557331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b7c9-b141-42f0-bb0f-4756b1a653ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:38.557944 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b7c9-b141-42f0-bb0f-4756b1a653ca None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:38.558219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b7c9-b141-42f0-bb0f-4756b1a653ca None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:38.558485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b7c9-b141-42f0-bb0f-4756b1a653ca None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:38.559163 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3b7c9-b141-42f0-bb0f-4756b1a653ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:02:38.559461 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3b7c9-b141-42f0-bb0f-4756b1a653ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:38.561234 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3b7c9-b141-42f0-bb0f-4756b1a653ca None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:38.569394 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 597/1196] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:02:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:38.578707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731c5b8-1c5e-4345-8447-1a0cb1a1ea86 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:38.664863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:38.684491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731c5b8-1c5e-4345-8447-1a0cb1a1ea86 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:38.724429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df3bd9-fa62-4520-808d-4cf0bc072da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:38.725104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df3bd9-fa62-4520-808d-4cf0bc072da6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:38.725378 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df3bd9-fa62-4520-808d-4cf0bc072da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:38.725645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df3bd9-fa62-4520-808d-4cf0bc072da6 None admin] PATH_INFO: `/v3/users/63566e537ba948fdbae7fd83529539c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:38.726787 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df3bd9-fa62-4520-808d-4cf0bc072da6 None admin] RBAC: Authorizing `identity:get_user(user_id=63566e537ba948fdbae7fd83529539c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:38.728234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df3bd9-fa62-4520-808d-4cf0bc072da6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:38.729832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 600/1197] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Sep 6 21:02:38 2022] GET /identity/v3/users/63566e537ba948fdbae7fd83529539c5 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:38.736124 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b7c9-b141-42f0-bb0f-4756b1a653ca None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:38.857953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b7c9-b141-42f0-bb0f-4756b1a653ca None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:38.900681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab54623-a430-45d8-b7dc-84cf7f754932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:38.901313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab54623-a430-45d8-b7dc-84cf7f754932 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:38.901592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab54623-a430-45d8-b7dc-84cf7f754932 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:38.901865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab54623-a430-45d8-b7dc-84cf7f754932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:38.961802 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:38.976323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab54623-a430-45d8-b7dc-84cf7f754932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:38.977812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab54623-a430-45d8-b7dc-84cf7f754932 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:39.100380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 598/1198] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:02:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:02:39.107528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df3bd9-fa62-4520-808d-4cf0bc072da6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:39.216087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df3bd9-fa62-4520-808d-4cf0bc072da6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:39.252671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eab811-2d1d-4fe8-883a-95bf0470d7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:39.253306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eab811-2d1d-4fe8-883a-95bf0470d7ce None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:39.253584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eab811-2d1d-4fe8-883a-95bf0470d7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:39.253855 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eab811-2d1d-4fe8-883a-95bf0470d7ce None admin] PATH_INFO: `/v3/projects/4e2b0fab14d44cb89a8b4eacfeb4f818` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:39.255022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eab811-2d1d-4fe8-883a-95bf0470d7ce None admin] RBAC: Authorizing `identity:get_project(project_id=4e2b0fab14d44cb89a8b4eacfeb4f818)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:39.256563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eab811-2d1d-4fe8-883a-95bf0470d7ce None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:39.258230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 601/1199] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Sep 6 21:02:39 2022] GET /identity/v3/projects/4e2b0fab14d44cb89a8b4eacfeb4f818 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:39.264681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab54623-a430-45d8-b7dc-84cf7f754932 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:02:39.365023 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:39.380646 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab54623-a430-45d8-b7dc-84cf7f754932 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:02:39.431845 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2b95b-4163-4707-ab88-4b00dbc1653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:02:39.432434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2b95b-4163-4707-ab88-4b00dbc1653f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:02:39.432726 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2b95b-4163-4707-ab88-4b00dbc1653f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:02:39.433014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2b95b-4163-4707-ab88-4b00dbc1653f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:02:39.433712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2b95b-4163-4707-ab88-4b00dbc1653f None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=4e2b0fab14d44cb89a8b4eacfeb4f818, user.id=63566e537ba948fdbae7fd83529539c5) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:02:39.434010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2b95b-4163-4707-ab88-4b00dbc1653f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:02:39.435805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2b95b-4163-4707-ab88-4b00dbc1653f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:02:39.441754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 599/1200] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:02:39 2022] GET /identity/v3/role_assignments?user.id=63566e537ba948fdbae7fd83529539c5&scope.project.id=4e2b0fab14d44cb89a8b4eacfeb4f818&role.id=3b30504991194002b41f8587d0851c10 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:02:49.248258 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:02:49.439315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=23 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:35.917690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ef0fb-5894-4402-bbf7-aba3fa72c4db None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:35.918274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ef0fb-5894-4402-bbf7-aba3fa72c4db None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:35.918274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ef0fb-5894-4402-bbf7-aba3fa72c4db None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:35.919479 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 602/1201] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:03:35.925600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa21f3-22ef-4a63-9a65-5d77ba2e168a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:35.926121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa21f3-22ef-4a63-9a65-5d77ba2e168a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:35.926252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa21f3-22ef-4a63-9a65-5d77ba2e168a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:35.948886 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b6fa21f3-22ef-4a63-9a65-5d77ba2e168a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:03:35.950138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b6fa21f3-22ef-4a63-9a65-5d77ba2e168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:03:36.039106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 600/1202] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:03:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:03:36.328339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ef0fb-5894-4402-bbf7-aba3fa72c4db None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:36.464712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ef0fb-5894-4402-bbf7-aba3fa72c4db None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:36.505487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fdc2e-2c8e-4a25-93c3-5fcc7049e642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:36.506135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fdc2e-2c8e-4a25-93c3-5fcc7049e642 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:36.506435 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fdc2e-2c8e-4a25-93c3-5fcc7049e642 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:36.506700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fdc2e-2c8e-4a25-93c3-5fcc7049e642 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:36.518334 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fdc2e-2c8e-4a25-93c3-5fcc7049e642 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:36.519919 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fdc2e-2c8e-4a25-93c3-5fcc7049e642 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9e6fdc2e-2c8e-4a25-93c3-5fcc7049e642 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:03:36.537779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:36.539467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:03:36.540681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:03:36.540681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:36.540681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:03:36.540681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 603/1203] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:36.548918 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa21f3-22ef-4a63-9a65-5d77ba2e168a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:36.694839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa21f3-22ef-4a63-9a65-5d77ba2e168a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:36.730568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:36.738317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4147869-e5bd-4c82-a5bf-84f8e826d276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:36.739152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4147869-e5bd-4c82-a5bf-84f8e826d276 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:36.739535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4147869-e5bd-4c82-a5bf-84f8e826d276 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:36.739788 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4147869-e5bd-4c82-a5bf-84f8e826d276 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:36.740775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4147869-e5bd-4c82-a5bf-84f8e826d276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:36.741258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4147869-e5bd-4c82-a5bf-84f8e826d276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:36.743041 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4147869-e5bd-4c82-a5bf-84f8e826d276 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:36.758052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 601/1204] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:36.764150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fdc2e-2c8e-4a25-93c3-5fcc7049e642 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:36.836738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:36.902070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fdc2e-2c8e-4a25-93c3-5fcc7049e642 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:36.951517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced24938-ca1b-4cfa-9b0b-fb4ce34b09df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:36.952206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced24938-ca1b-4cfa-9b0b-fb4ce34b09df None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:36.952560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced24938-ca1b-4cfa-9b0b-fb4ce34b09df None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:36.952880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced24938-ca1b-4cfa-9b0b-fb4ce34b09df None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:36.953649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced24938-ca1b-4cfa-9b0b-fb4ce34b09df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:36.955153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced24938-ca1b-4cfa-9b0b-fb4ce34b09df None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:36.980425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 604/1205] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Sep 6 21:03:36 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:03:39.416647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aefbc6-8826-4ddd-8506-ddda5b28827b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:39.417260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aefbc6-8826-4ddd-8506-ddda5b28827b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:39.417260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aefbc6-8826-4ddd-8506-ddda5b28827b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:39.418576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 602/1206] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:03:39.424316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140655c7-ad95-4081-bbff-d1467df943ed None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:39.424770 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140655c7-ad95-4081-bbff-d1467df943ed None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:39.424873 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140655c7-ad95-4081-bbff-d1467df943ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:39.445536 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-140655c7-ad95-4081-bbff-d1467df943ed None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:03:39.446771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-140655c7-ad95-4081-bbff-d1467df943ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:03:39.521003 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 605/1207] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:03:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:03:39.794045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aefbc6-8826-4ddd-8506-ddda5b28827b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:39.925043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aefbc6-8826-4ddd-8506-ddda5b28827b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:39.970003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f236e-7537-4dda-9c62-5b90bb899c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:39.970802 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f236e-7537-4dda-9c62-5b90bb899c61 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:39.971086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f236e-7537-4dda-9c62-5b90bb899c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:39.971390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f236e-7537-4dda-9c62-5b90bb899c61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:39.983648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f236e-7537-4dda-9c62-5b90bb899c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b24f236e-7537-4dda-9c62-5b90bb899c61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:03:39.989102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:39.990207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:39.991059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:39.991059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:39.991059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:39.991059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:39.991059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:03:39.991059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:03:39.991059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:03:39.991059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:03:39.991059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:03:39.991059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:03:39.991059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:03:39.991059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:03:39.991059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 603/1208] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:03:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:40.000267 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140655c7-ad95-4081-bbff-d1467df943ed None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:40.044983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:40.123484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140655c7-ad95-4081-bbff-d1467df943ed None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:40.166622 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a997d-1bb1-4251-8875-c65826bd8f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:40.167265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a997d-1bb1-4251-8875-c65826bd8f7b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:40.167528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a997d-1bb1-4251-8875-c65826bd8f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:40.167911 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a997d-1bb1-4251-8875-c65826bd8f7b None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:40.168517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a997d-1bb1-4251-8875-c65826bd8f7b None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:40.168888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a997d-1bb1-4251-8875-c65826bd8f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:40.170350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a997d-1bb1-4251-8875-c65826bd8f7b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:40.179343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 606/1209] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:03:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:40.187165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f236e-7537-4dda-9c62-5b90bb899c61 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:40.256418 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:40.315273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f236e-7537-4dda-9c62-5b90bb899c61 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:40.358489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904686fb-b60d-4bca-b488-516ba5101c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:40.359439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904686fb-b60d-4bca-b488-516ba5101c73 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:40.359994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904686fb-b60d-4bca-b488-516ba5101c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:40.360302 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904686fb-b60d-4bca-b488-516ba5101c73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:40.372720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904686fb-b60d-4bca-b488-516ba5101c73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:40.374541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904686fb-b60d-4bca-b488-516ba5101c73 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-904686fb-b60d-4bca-b488-516ba5101c73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:03:40.390534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:40.391753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:03:40.392886 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:03:40.392886 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:40.392886 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:03:40.393991 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 604/1210] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:40.400327 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a997d-1bb1-4251-8875-c65826bd8f7b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:40.511348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:40.515167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a997d-1bb1-4251-8875-c65826bd8f7b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:40.555149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683d2de-c724-4588-8fd7-938e6aab9656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:40.556115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683d2de-c724-4588-8fd7-938e6aab9656 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:40.556390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683d2de-c724-4588-8fd7-938e6aab9656 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:40.556899 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683d2de-c724-4588-8fd7-938e6aab9656 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:40.557344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683d2de-c724-4588-8fd7-938e6aab9656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:40.557692 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683d2de-c724-4588-8fd7-938e6aab9656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:40.559009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683d2de-c724-4588-8fd7-938e6aab9656 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:40.577165 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 607/1211] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:03:40.578475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904686fb-b60d-4bca-b488-516ba5101c73 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:40.714271 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904686fb-b60d-4bca-b488-516ba5101c73 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:40.715998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:40.753525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabe29d-f3bd-43b9-bcbc-ec80c2c68bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:40.754211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabe29d-f3bd-43b9-bcbc-ec80c2c68bef None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:40.754491 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabe29d-f3bd-43b9-bcbc-ec80c2c68bef None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:40.754780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabe29d-f3bd-43b9-bcbc-ec80c2c68bef None admin] PATH_INFO: `/v3/users/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:40.763201 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabe29d-f3bd-43b9-bcbc-ec80c2c68bef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:40.764726 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabe29d-f3bd-43b9-bcbc-ec80c2c68bef None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-bfabe29d-f3bd-43b9-bcbc-ec80c2c68bef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:03:40.778104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:40.779122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:40.780075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:40.780075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:40.780075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:03:40.780075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:03:40.780075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:03:40.780075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:03:40.780075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:03:40.780075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:03:40.780075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 21:03:40.780075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:03:40.780075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 605/1212] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:03:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:40.785620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683d2de-c724-4588-8fd7-938e6aab9656 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:40.904803 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683d2de-c724-4588-8fd7-938e6aab9656 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:40.941486 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0845f9-816d-48dd-9c1d-df844534f061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:40.942171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0845f9-816d-48dd-9c1d-df844534f061 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:40.942505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0845f9-816d-48dd-9c1d-df844534f061 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:40.942737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0845f9-816d-48dd-9c1d-df844534f061 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:40.943498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0845f9-816d-48dd-9c1d-df844534f061 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:40.943806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0845f9-816d-48dd-9c1d-df844534f061 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:40.945453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0845f9-816d-48dd-9c1d-df844534f061 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:40.971927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 608/1213] 10.222.0.79 () {56 vars in 1143 bytes} [Tue Sep 6 21:03:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:40.973786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabe29d-f3bd-43b9-bcbc-ec80c2c68bef None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:41.089250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabe29d-f3bd-43b9-bcbc-ec80c2c68bef None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:41.125554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933bb72-b925-423d-a14f-8e436f26b8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:41.126164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933bb72-b925-423d-a14f-8e436f26b8dc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:41.126441 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933bb72-b925-423d-a14f-8e436f26b8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:41.126707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933bb72-b925-423d-a14f-8e436f26b8dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:41.160432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:41.206061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933bb72-b925-423d-a14f-8e436f26b8dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:41.207705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933bb72-b925-423d-a14f-8e436f26b8dc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:41.328999 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 606/1214] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:03:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:03:41.330210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0845f9-816d-48dd-9c1d-df844534f061 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:41.351157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:41.448749 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0845f9-816d-48dd-9c1d-df844534f061 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:41.492465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c54f78-1b1b-4960-8ddc-65feb1ec01a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:41.493395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54f78-1b1b-4960-8ddc-65feb1ec01a0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:41.493683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54f78-1b1b-4960-8ddc-65feb1ec01a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:41.493983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54f78-1b1b-4960-8ddc-65feb1ec01a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:41.505607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c54f78-1b1b-4960-8ddc-65feb1ec01a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:41.507136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c54f78-1b1b-4960-8ddc-65feb1ec01a0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c0c54f78-1b1b-4960-8ddc-65feb1ec01a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:03:41.523498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:41.524997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:03:41.525903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:03:41.525903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:41.525903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:03:41.525903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 609/1215] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:41.532616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933bb72-b925-423d-a14f-8e436f26b8dc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:41.605234 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:41.651045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933bb72-b925-423d-a14f-8e436f26b8dc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:41.698165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3639946-f2fe-4f7d-9d8a-e8bf23e6c39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:41.698830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3639946-f2fe-4f7d-9d8a-e8bf23e6c39f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:41.699053 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3639946-f2fe-4f7d-9d8a-e8bf23e6c39f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:41.699322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3639946-f2fe-4f7d-9d8a-e8bf23e6c39f None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:41.699969 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3639946-f2fe-4f7d-9d8a-e8bf23e6c39f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:41.700265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3639946-f2fe-4f7d-9d8a-e8bf23e6c39f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:41.701677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3639946-f2fe-4f7d-9d8a-e8bf23e6c39f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:41.714356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 607/1216] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:41.720484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c54f78-1b1b-4960-8ddc-65feb1ec01a0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:41.799319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:41.835997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c54f78-1b1b-4960-8ddc-65feb1ec01a0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:41.874061 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef4855-ebf7-458b-b29f-5619784ebc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:41.874737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef4855-ebf7-458b-b29f-5619784ebc0f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:41.875038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef4855-ebf7-458b-b29f-5619784ebc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:41.875332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef4855-ebf7-458b-b29f-5619784ebc0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:41.882453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef4855-ebf7-458b-b29f-5619784ebc0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:41.883930 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef4855-ebf7-458b-b29f-5619784ebc0f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-46ef4855-ebf7-458b-b29f-5619784ebc0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:03:41.896431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:41.897690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:41.898656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:03:41.898656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:03:41.898656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:03:41.898656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:03:41.898656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:03:41.898656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:03:41.898656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:03:41.898656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:03:41.898656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 610/1217] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:03:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:41.903573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3639946-f2fe-4f7d-9d8a-e8bf23e6c39f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:42.014445 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3639946-f2fe-4f7d-9d8a-e8bf23e6c39f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:42.039864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:42.054335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15385c6d-fe7a-4ded-8c27-5d4ec41030bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:42.054971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15385c6d-fe7a-4ded-8c27-5d4ec41030bf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:42.055288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15385c6d-fe7a-4ded-8c27-5d4ec41030bf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:42.055568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15385c6d-fe7a-4ded-8c27-5d4ec41030bf None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:42.056473 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15385c6d-fe7a-4ded-8c27-5d4ec41030bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:42.056997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15385c6d-fe7a-4ded-8c27-5d4ec41030bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:42.058745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15385c6d-fe7a-4ded-8c27-5d4ec41030bf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:42.077797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 608/1218] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:03:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:42.078981 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef4855-ebf7-458b-b29f-5619784ebc0f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:42.193874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef4855-ebf7-458b-b29f-5619784ebc0f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:42.223971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:42.230800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078c6b2-d0e5-4283-a854-a142aac30ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:42.231464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078c6b2-d0e5-4283-a854-a142aac30ebe None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:42.231756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078c6b2-d0e5-4283-a854-a142aac30ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:42.232090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078c6b2-d0e5-4283-a854-a142aac30ebe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:42.232876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078c6b2-d0e5-4283-a854-a142aac30ebe None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=c40b5ff21d1940749bf479cc936fc0f2) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:42.233205 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078c6b2-d0e5-4283-a854-a142aac30ebe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:42.234839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078c6b2-d0e5-4283-a854-a142aac30ebe None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:42.243998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 611/1219] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:03:42 2022] GET /identity/v3/role_assignments?user.id=c40b5ff21d1940749bf479cc936fc0f2&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:44.487203 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e4eed-6182-48ca-aa88-dcb820c4685b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:44.487808 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e4eed-6182-48ca-aa88-dcb820c4685b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:44.487808 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e4eed-6182-48ca-aa88-dcb820c4685b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:44.489224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 609/1220] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:03:44.494878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751849fe-c099-486a-a8eb-5678c84c881d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:44.495331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751849fe-c099-486a-a8eb-5678c84c881d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:44.495422 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751849fe-c099-486a-a8eb-5678c84c881d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:44.517102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-751849fe-c099-486a-a8eb-5678c84c881d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:03:44.518334 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-751849fe-c099-486a-a8eb-5678c84c881d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:03:44.599497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 612/1221] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:03:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:03:44.872857 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e4eed-6182-48ca-aa88-dcb820c4685b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:44.986158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e4eed-6182-48ca-aa88-dcb820c4685b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:45.023961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330d463-104b-4ca1-947a-24d97c373c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:45.024696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330d463-104b-4ca1-947a-24d97c373c54 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:45.024951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330d463-104b-4ca1-947a-24d97c373c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:45.025222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330d463-104b-4ca1-947a-24d97c373c54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:45.035670 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330d463-104b-4ca1-947a-24d97c373c54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b330d463-104b-4ca1-947a-24d97c373c54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:03:45.041128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:45.042376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:45.043287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:45.043287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:45.043287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:45.043287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:45.043287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:03:45.043287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:03:45.043287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:03:45.043287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:03:45.043287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:03:45.043287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:03:45.043287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:03:45.043287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:03:45.043287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 610/1222] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:03:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:45.048682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751849fe-c099-486a-a8eb-5678c84c881d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:45.158870 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751849fe-c099-486a-a8eb-5678c84c881d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:45.194310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:45.208081 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef4d1b-2a8b-4189-87f9-a8e8b5afbfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:45.208897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef4d1b-2a8b-4189-87f9-a8e8b5afbfe8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:45.209289 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef4d1b-2a8b-4189-87f9-a8e8b5afbfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:45.209615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef4d1b-2a8b-4189-87f9-a8e8b5afbfe8 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:45.210599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef4d1b-2a8b-4189-87f9-a8e8b5afbfe8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:45.211014 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef4d1b-2a8b-4189-87f9-a8e8b5afbfe8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:45.212878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef4d1b-2a8b-4189-87f9-a8e8b5afbfe8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:45.224475 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 613/1223] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:03:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:45.231805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330d463-104b-4ca1-947a-24d97c373c54 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:45.283344 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:45.358219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330d463-104b-4ca1-947a-24d97c373c54 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:45.401460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc2738-97c9-422c-9af8-90f9f68f236b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:45.402502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc2738-97c9-422c-9af8-90f9f68f236b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:45.402935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc2738-97c9-422c-9af8-90f9f68f236b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:45.403383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc2738-97c9-422c-9af8-90f9f68f236b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:45.419178 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc2738-97c9-422c-9af8-90f9f68f236b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:45.420975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc2738-97c9-422c-9af8-90f9f68f236b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-09bc2738-97c9-422c-9af8-90f9f68f236b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:03:45.437502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:45.439008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:03:45.440098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:03:45.440098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:45.440098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:03:45.440098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 611/1224] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:45.445625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef4d1b-2a8b-4189-87f9-a8e8b5afbfe8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:45.557685 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef4d1b-2a8b-4189-87f9-a8e8b5afbfe8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:45.601942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23751ace-07e8-4625-9cd4-045443a3d256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:45.602695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23751ace-07e8-4625-9cd4-045443a3d256 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:45.603108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23751ace-07e8-4625-9cd4-045443a3d256 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:45.603393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23751ace-07e8-4625-9cd4-045443a3d256 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:45.604050 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23751ace-07e8-4625-9cd4-045443a3d256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:45.604355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23751ace-07e8-4625-9cd4-045443a3d256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:45.605786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23751ace-07e8-4625-9cd4-045443a3d256 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:45.618438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 614/1225] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:45.624795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc2738-97c9-422c-9af8-90f9f68f236b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:45.739041 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:45.746851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc2738-97c9-422c-9af8-90f9f68f236b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:45.795268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3161c8-d62b-4d89-82b9-7c34bfc13e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:45.795949 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3161c8-d62b-4d89-82b9-7c34bfc13e97 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:45.796227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3161c8-d62b-4d89-82b9-7c34bfc13e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:45.796519 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3161c8-d62b-4d89-82b9-7c34bfc13e97 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:45.805109 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3161c8-d62b-4d89-82b9-7c34bfc13e97 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:45.806700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3161c8-d62b-4d89-82b9-7c34bfc13e97 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7f3161c8-d62b-4d89-82b9-7c34bfc13e97 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:03:45.821480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:45.822535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:45.823747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:45.823747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:45.823747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:03:45.823747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:03:45.823747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:03:45.823747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:03:45.823747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:03:45.823747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:03:45.823747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 21:03:45.823747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:03:45.823747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 612/1226] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:03:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:45.829742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23751ace-07e8-4625-9cd4-045443a3d256 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:45.887172 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:45.945820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23751ace-07e8-4625-9cd4-045443a3d256 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:45.983427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85501fb9-c757-4249-a9b5-71f27cc791f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:45.984030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85501fb9-c757-4249-a9b5-71f27cc791f0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:45.984319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85501fb9-c757-4249-a9b5-71f27cc791f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:45.984567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85501fb9-c757-4249-a9b5-71f27cc791f0 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:45.985319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85501fb9-c757-4249-a9b5-71f27cc791f0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:45.985625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85501fb9-c757-4249-a9b5-71f27cc791f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:45.987091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85501fb9-c757-4249-a9b5-71f27cc791f0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:46.006672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 615/1227] 10.222.0.79 () {56 vars in 1143 bytes} [Tue Sep 6 21:03:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:03:46.013629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3161c8-d62b-4d89-82b9-7c34bfc13e97 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:46.150726 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3161c8-d62b-4d89-82b9-7c34bfc13e97 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:46.188099 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c2c77-ddc9-409a-9031-2cbf69494a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:46.188860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c2c77-ddc9-409a-9031-2cbf69494a85 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:46.189134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c2c77-ddc9-409a-9031-2cbf69494a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:46.189396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c2c77-ddc9-409a-9031-2cbf69494a85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:46.200715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c2c77-ddc9-409a-9031-2cbf69494a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:46.202181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c2c77-ddc9-409a-9031-2cbf69494a85 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-fd8c2c77-ddc9-409a-9031-2cbf69494a85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:03:46.218771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:46.219967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:03:46.221033 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:03:46.221033 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:46.221033 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:03:46.221033 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 613/1228] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:03:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:03:46.226483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85501fb9-c757-4249-a9b5-71f27cc791f0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:46.347839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85501fb9-c757-4249-a9b5-71f27cc791f0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:46.350622 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:46.389851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41af0e6-efb4-40a9-8f14-3205f9e243fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:46.390461 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41af0e6-efb4-40a9-8f14-3205f9e243fb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:46.390732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41af0e6-efb4-40a9-8f14-3205f9e243fb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:46.390998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41af0e6-efb4-40a9-8f14-3205f9e243fb None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:46.391643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41af0e6-efb4-40a9-8f14-3205f9e243fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:46.391937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41af0e6-efb4-40a9-8f14-3205f9e243fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:46.393438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41af0e6-efb4-40a9-8f14-3205f9e243fb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:46.404939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 616/1229] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:46.411514 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c2c77-ddc9-409a-9031-2cbf69494a85 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:46.421986 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:46.532554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c2c77-ddc9-409a-9031-2cbf69494a85 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:46.580125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed5bf5-1cc2-403c-a7d1-786a4c68cd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:46.580791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed5bf5-1cc2-403c-a7d1-786a4c68cd2f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:46.581064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed5bf5-1cc2-403c-a7d1-786a4c68cd2f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:46.581326 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed5bf5-1cc2-403c-a7d1-786a4c68cd2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:46.591971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed5bf5-1cc2-403c-a7d1-786a4c68cd2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:46.593521 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed5bf5-1cc2-403c-a7d1-786a4c68cd2f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-25ed5bf5-1cc2-403c-a7d1-786a4c68cd2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:03:46.607743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:46.612748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:46.613947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:03:46.613947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:03:46.613947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:03:46.613947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:03:46.613947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:03:46.613947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:03:46.613947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:03:46.613947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:03:46.613947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 614/1230] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:03:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:46.614761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41af0e6-efb4-40a9-8f14-3205f9e243fb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:46.724890 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41af0e6-efb4-40a9-8f14-3205f9e243fb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:46.760283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ce8f4-ca69-4c5a-bbf1-04dac1e514d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:46.761073 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ce8f4-ca69-4c5a-bbf1-04dac1e514d1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:46.761355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ce8f4-ca69-4c5a-bbf1-04dac1e514d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:46.761624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ce8f4-ca69-4c5a-bbf1-04dac1e514d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:46.762480 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ce8f4-ca69-4c5a-bbf1-04dac1e514d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:46.762785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ce8f4-ca69-4c5a-bbf1-04dac1e514d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:46.765607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ce8f4-ca69-4c5a-bbf1-04dac1e514d1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:46.778897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 617/1231] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:03:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:46.785686 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed5bf5-1cc2-403c-a7d1-786a4c68cd2f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:46.872347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:46.912088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed5bf5-1cc2-403c-a7d1-786a4c68cd2f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:46.953893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9a40b-b814-4744-bfe0-d9ba8e45250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:46.954626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9a40b-b814-4744-bfe0-d9ba8e45250f None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:46.954992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9a40b-b814-4744-bfe0-d9ba8e45250f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:46.955469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9a40b-b814-4744-bfe0-d9ba8e45250f None admin] PATH_INFO: `/v3/projects/05af249ba0a047fe83aefb078e203e53/users/c40b5ff21d1940749bf479cc936fc0f2/roles/b3f49281e89f4fa1a2d119e2fb4922e5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:46.973504 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9a40b-b814-4744-bfe0-d9ba8e45250f None admin] RBAC: Authorizing `identity:create_grant(project_id=05af249ba0a047fe83aefb078e203e53, user_id=c40b5ff21d1940749bf479cc936fc0f2, role_id=b3f49281e89f4fa1a2d119e2fb4922e5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:46.975056 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9a40b-b814-4744-bfe0-d9ba8e45250f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:46.989164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 615/1232] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:03:46 2022] PUT /identity/v3/projects/05af249ba0a047fe83aefb078e203e53/users/c40b5ff21d1940749bf479cc936fc0f2/roles/b3f49281e89f4fa1a2d119e2fb4922e5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:03:49.157291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c4147-cf63-4c20-9600-bb2a5ab57a49 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:49.157864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c4147-cf63-4c20-9600-bb2a5ab57a49 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:49.157864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c4147-cf63-4c20-9600-bb2a5ab57a49 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:49.159187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 618/1233] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:03:49.164910 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b5331-db3a-4aab-a52b-d7e6b9a27f67 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:49.165357 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b5331-db3a-4aab-a52b-d7e6b9a27f67 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:49.165465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b5331-db3a-4aab-a52b-d7e6b9a27f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:49.189133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-479b5331-db3a-4aab-a52b-d7e6b9a27f67 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:03:49.191020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-479b5331-db3a-4aab-a52b-d7e6b9a27f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:03:49.264495 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 616/1234] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:03:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 21:03:49.542316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c4147-cf63-4c20-9600-bb2a5ab57a49 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:49.570419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:49.656637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c4147-cf63-4c20-9600-bb2a5ab57a49 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:49.696221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8bbc5-1dee-48c9-878a-372825ae8054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:49.696902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8bbc5-1dee-48c9-878a-372825ae8054 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:49.697191 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8bbc5-1dee-48c9-878a-372825ae8054 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:49.697508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8bbc5-1dee-48c9-878a-372825ae8054 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:49.708253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8bbc5-1dee-48c9-878a-372825ae8054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c4f8bbc5-1dee-48c9-878a-372825ae8054 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:03:49.713309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:49.714507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:49.715704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:49.715704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:49.715704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:49.715704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:49.715704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:03:49.715704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:03:49.715704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:03:49.715704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:03:49.715704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:03:49.715704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:03:49.715704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 21:03:49.715704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:03:49.715704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 619/1235] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Sep 6 21:03:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:49.720943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b5331-db3a-4aab-a52b-d7e6b9a27f67 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:49.770242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:49.826996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b5331-db3a-4aab-a52b-d7e6b9a27f67 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:49.866977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ab08a-5d5d-4b25-ba02-6db0c22dfa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:49.868018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ab08a-5d5d-4b25-ba02-6db0c22dfa0b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:49.868310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ab08a-5d5d-4b25-ba02-6db0c22dfa0b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:49.868580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ab08a-5d5d-4b25-ba02-6db0c22dfa0b None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:49.869362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ab08a-5d5d-4b25-ba02-6db0c22dfa0b None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:49.869717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ab08a-5d5d-4b25-ba02-6db0c22dfa0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:49.871334 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ab08a-5d5d-4b25-ba02-6db0c22dfa0b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:49.881331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 617/1236] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Sep 6 21:03:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:49.887650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8bbc5-1dee-48c9-878a-372825ae8054 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:49.983483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:50.000933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8bbc5-1dee-48c9-878a-372825ae8054 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:50.039615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b778cc2-cf5b-4e5c-bddd-65e0bdc1cb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:50.040257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778cc2-cf5b-4e5c-bddd-65e0bdc1cb40 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:50.040530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778cc2-cf5b-4e5c-bddd-65e0bdc1cb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:50.040830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778cc2-cf5b-4e5c-bddd-65e0bdc1cb40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:50.052205 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b778cc2-cf5b-4e5c-bddd-65e0bdc1cb40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:50.053832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b778cc2-cf5b-4e5c-bddd-65e0bdc1cb40 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0b778cc2-cf5b-4e5c-bddd-65e0bdc1cb40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:03:50.072834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:50.074092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:03:50.075035 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:03:50.075035 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:50.075035 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:03:50.075035 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 620/1237] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:50.080138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ab08a-5d5d-4b25-ba02-6db0c22dfa0b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:50.195699 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ab08a-5d5d-4b25-ba02-6db0c22dfa0b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:50.206863 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:50.238241 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738b127-8e63-4af1-898c-8a3a2456a896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:50.238919 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738b127-8e63-4af1-898c-8a3a2456a896 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:50.239195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738b127-8e63-4af1-898c-8a3a2456a896 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:50.239465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738b127-8e63-4af1-898c-8a3a2456a896 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:50.240180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738b127-8e63-4af1-898c-8a3a2456a896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:50.240445 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738b127-8e63-4af1-898c-8a3a2456a896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:50.241874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738b127-8e63-4af1-898c-8a3a2456a896 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:50.254856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 618/1238] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:03:50.260788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b778cc2-cf5b-4e5c-bddd-65e0bdc1cb40 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:50.372205 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b778cc2-cf5b-4e5c-bddd-65e0bdc1cb40 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:50.410172 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048dc2c-15ca-4929-91f3-8694e41020a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:50.410999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048dc2c-15ca-4929-91f3-8694e41020a2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:50.411292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048dc2c-15ca-4929-91f3-8694e41020a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:50.411579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048dc2c-15ca-4929-91f3-8694e41020a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:50.414631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:50.420319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048dc2c-15ca-4929-91f3-8694e41020a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:50.422051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048dc2c-15ca-4929-91f3-8694e41020a2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f048dc2c-15ca-4929-91f3-8694e41020a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:03:50.435843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:50.437078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:50.438135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:50.438135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:50.438135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:03:50.438135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:03:50.438135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:03:50.438135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:03:50.438135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:03:50.438135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:03:50.438135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 21:03:50.438135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:03:50.438135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 621/1239] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:03:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:50.443629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738b127-8e63-4af1-898c-8a3a2456a896 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:50.561598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738b127-8e63-4af1-898c-8a3a2456a896 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:50.605446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b93d9-31e4-4273-bcf0-302b7c464a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:50.606255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b93d9-31e4-4273-bcf0-302b7c464a09 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:50.606567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b93d9-31e4-4273-bcf0-302b7c464a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:50.606902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b93d9-31e4-4273-bcf0-302b7c464a09 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:50.607902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b93d9-31e4-4273-bcf0-302b7c464a09 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:50.608209 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b93d9-31e4-4273-bcf0-302b7c464a09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:50.609689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b93d9-31e4-4273-bcf0-302b7c464a09 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:50.630639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 619/1240] 10.222.0.79 () {56 vars in 1143 bytes} [Tue Sep 6 21:03:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:03:50.636926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048dc2c-15ca-4929-91f3-8694e41020a2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:50.751568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048dc2c-15ca-4929-91f3-8694e41020a2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:50.788132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746d48f-8b81-4dd4-86ac-129587f4dab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:50.788893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746d48f-8b81-4dd4-86ac-129587f4dab3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:50.789089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746d48f-8b81-4dd4-86ac-129587f4dab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:50.789348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746d48f-8b81-4dd4-86ac-129587f4dab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:50.856126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:50.864315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746d48f-8b81-4dd4-86ac-129587f4dab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:50.866019 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746d48f-8b81-4dd4-86ac-129587f4dab3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:50.977924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 622/1241] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:03:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:03:50.983930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b93d9-31e4-4273-bcf0-302b7c464a09 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:51.013471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:51.100748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b93d9-31e4-4273-bcf0-302b7c464a09 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:51.139883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96677153-3196-4300-8562-9f1f8369412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:51.140645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96677153-3196-4300-8562-9f1f8369412e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:51.141005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96677153-3196-4300-8562-9f1f8369412e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:51.141276 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96677153-3196-4300-8562-9f1f8369412e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:51.152832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96677153-3196-4300-8562-9f1f8369412e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:51.154290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96677153-3196-4300-8562-9f1f8369412e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-96677153-3196-4300-8562-9f1f8369412e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:03:51.170240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:51.171320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:03:51.172412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:03:51.172412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:51.172412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:03:51.172412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 620/1242] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:51.177499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746d48f-8b81-4dd4-86ac-129587f4dab3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:51.281984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:51.290694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746d48f-8b81-4dd4-86ac-129587f4dab3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:51.330484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e75fe-a8d2-42a4-945f-819ba5540408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:51.331196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e75fe-a8d2-42a4-945f-819ba5540408 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:51.331540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e75fe-a8d2-42a4-945f-819ba5540408 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:51.331871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e75fe-a8d2-42a4-945f-819ba5540408 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:51.332877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e75fe-a8d2-42a4-945f-819ba5540408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:51.333182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e75fe-a8d2-42a4-945f-819ba5540408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:51.334707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e75fe-a8d2-42a4-945f-819ba5540408 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:51.347772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 623/1243] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:51.353797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96677153-3196-4300-8562-9f1f8369412e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:51.437581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:51.461794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96677153-3196-4300-8562-9f1f8369412e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:51.503343 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3db88a-d13b-4f55-85d4-7c6b6801185b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:51.503983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3db88a-d13b-4f55-85d4-7c6b6801185b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:51.504320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3db88a-d13b-4f55-85d4-7c6b6801185b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:51.504593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3db88a-d13b-4f55-85d4-7c6b6801185b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:51.512776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3db88a-d13b-4f55-85d4-7c6b6801185b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:51.514450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3db88a-d13b-4f55-85d4-7c6b6801185b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9d3db88a-d13b-4f55-85d4-7c6b6801185b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:03:51.526568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:51.527517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:51.528403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:03:51.528403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:03:51.528403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:03:51.528403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:03:51.528403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:03:51.528403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:03:51.528403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:03:51.528403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:03:51.528403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 621/1244] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:03:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:51.533987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e75fe-a8d2-42a4-945f-819ba5540408 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:51.644492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e75fe-a8d2-42a4-945f-819ba5540408 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:51.684218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab528e-75e7-4888-aef4-4a7d050367ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:51.685257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab528e-75e7-4888-aef4-4a7d050367ac None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:51.685664 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab528e-75e7-4888-aef4-4a7d050367ac None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:51.685947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab528e-75e7-4888-aef4-4a7d050367ac None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:51.686672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab528e-75e7-4888-aef4-4a7d050367ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:51.686989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab528e-75e7-4888-aef4-4a7d050367ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:51.688752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab528e-75e7-4888-aef4-4a7d050367ac None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:51.702645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 624/1245] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:03:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:51.708955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3db88a-d13b-4f55-85d4-7c6b6801185b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:51.823031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3db88a-d13b-4f55-85d4-7c6b6801185b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:51.866463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f79542-6579-4e85-becb-474cb0692922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:51.867463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f79542-6579-4e85-becb-474cb0692922 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:51.867901 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f79542-6579-4e85-becb-474cb0692922 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:51.868295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f79542-6579-4e85-becb-474cb0692922 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:51.869222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f79542-6579-4e85-becb-474cb0692922 None admin] RBAC: Adding query filter params (role.id=b3f49281e89f4fa1a2d119e2fb4922e5, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=c40b5ff21d1940749bf479cc936fc0f2) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:51.869531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f79542-6579-4e85-becb-474cb0692922 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:51.871314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f79542-6579-4e85-becb-474cb0692922 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:51.878802 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 622/1246] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:03:51 2022] GET /identity/v3/role_assignments?user.id=c40b5ff21d1940749bf479cc936fc0f2&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=b3f49281e89f4fa1a2d119e2fb4922e5 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:54.387003 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1a011-7023-43bd-b771-b7615877d1df None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:54.387573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1a011-7023-43bd-b771-b7615877d1df None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:54.387573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1a011-7023-43bd-b771-b7615877d1df None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:54.389221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 625/1247] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:03:54.395334 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c0008-c285-4894-b3e5-72f66caf2443 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:54.395899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c0008-c285-4894-b3e5-72f66caf2443 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:54.395899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c0008-c285-4894-b3e5-72f66caf2443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:54.403120 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:54.418619 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a00c0008-c285-4894-b3e5-72f66caf2443 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:03:54.420632 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a00c0008-c285-4894-b3e5-72f66caf2443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:03:54.507517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 623/1248] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:03:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:03:54.803342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1a011-7023-43bd-b771-b7615877d1df None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:54.830534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:54.937929 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1a011-7023-43bd-b771-b7615877d1df None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:54.978704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f28a2-0589-4e31-a387-d8f6a7a6ccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:54.979397 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f28a2-0589-4e31-a387-d8f6a7a6ccb7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:54.979678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f28a2-0589-4e31-a387-d8f6a7a6ccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:54.979949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f28a2-0589-4e31-a387-d8f6a7a6ccb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:54.993580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f28a2-0589-4e31-a387-d8f6a7a6ccb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-508f28a2-0589-4e31-a387-d8f6a7a6ccb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:03:54.999579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:55.000607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:55.001532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:55.001532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:55.001532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:55.001532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:55.001532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:03:55.001532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:03:55.001532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:03:55.001532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:03:55.001532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:03:55.001532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:03:55.001532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:03:55.001532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:03:55.001532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 626/1249] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:03:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:55.007750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c0008-c285-4894-b3e5-72f66caf2443 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:55.126425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c0008-c285-4894-b3e5-72f66caf2443 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:55.162679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:55.168701 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156df74f-7d69-4155-9d44-a17f4cd5da49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:55.169322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156df74f-7d69-4155-9d44-a17f4cd5da49 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:55.169609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156df74f-7d69-4155-9d44-a17f4cd5da49 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:55.169876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156df74f-7d69-4155-9d44-a17f4cd5da49 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:55.170593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156df74f-7d69-4155-9d44-a17f4cd5da49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:55.170995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156df74f-7d69-4155-9d44-a17f4cd5da49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:55.172557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156df74f-7d69-4155-9d44-a17f4cd5da49 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:55.181419 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 624/1250] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:03:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:55.188059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f28a2-0589-4e31-a387-d8f6a7a6ccb7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:55.292467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:55.316529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f28a2-0589-4e31-a387-d8f6a7a6ccb7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:55.357410 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd31bc-7c99-499d-9d58-b5506f606663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:55.358074 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd31bc-7c99-499d-9d58-b5506f606663 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:55.358348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd31bc-7c99-499d-9d58-b5506f606663 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:55.358585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd31bc-7c99-499d-9d58-b5506f606663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:55.374861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd31bc-7c99-499d-9d58-b5506f606663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:55.376934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd31bc-7c99-499d-9d58-b5506f606663 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-dffd31bc-7c99-499d-9d58-b5506f606663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:03:55.396132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:55.397827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:03:55.398727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:03:55.398727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:55.398727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:03:55.398727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 627/1251] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:55.408925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156df74f-7d69-4155-9d44-a17f4cd5da49 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:55.520914 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156df74f-7d69-4155-9d44-a17f4cd5da49 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:55.556097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38f488-8835-423b-a99e-494bdb335485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:55.556729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38f488-8835-423b-a99e-494bdb335485 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:55.557002 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38f488-8835-423b-a99e-494bdb335485 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:55.557268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38f488-8835-423b-a99e-494bdb335485 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:55.557905 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38f488-8835-423b-a99e-494bdb335485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:55.558200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38f488-8835-423b-a99e-494bdb335485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:55.559928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38f488-8835-423b-a99e-494bdb335485 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:55.574405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 625/1252] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:55.580321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd31bc-7c99-499d-9d58-b5506f606663 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:55.700359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd31bc-7c99-499d-9d58-b5506f606663 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:55.732560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:55.741750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa6e2f-6ac5-419e-a520-a0435b2de8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:55.742508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa6e2f-6ac5-419e-a520-a0435b2de8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:55.742809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa6e2f-6ac5-419e-a520-a0435b2de8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:55.743116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa6e2f-6ac5-419e-a520-a0435b2de8e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:55.751746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa6e2f-6ac5-419e-a520-a0435b2de8e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:55.753258 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa6e2f-6ac5-419e-a520-a0435b2de8e1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1caa6e2f-6ac5-419e-a520-a0435b2de8e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:03:55.765969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:55.767184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:55.768337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:55.768337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:55.768337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:03:55.768337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:03:55.768337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:03:55.768337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:03:55.768337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:03:55.768337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:03:55.768337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 21:03:55.768337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:03:55.768337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 628/1253] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:03:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:55.773215 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38f488-8835-423b-a99e-494bdb335485 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:55.848602 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:55.918646 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38f488-8835-423b-a99e-494bdb335485 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:55.964530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d34d2-fc9d-4c03-969b-b41249c59e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:55.965257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d34d2-fc9d-4c03-969b-b41249c59e45 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:55.965621 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d34d2-fc9d-4c03-969b-b41249c59e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:55.965998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d34d2-fc9d-4c03-969b-b41249c59e45 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:55.966818 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d34d2-fc9d-4c03-969b-b41249c59e45 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:55.967211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d34d2-fc9d-4c03-969b-b41249c59e45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:55.968782 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d34d2-fc9d-4c03-969b-b41249c59e45 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:55.989883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 626/1254] 10.222.0.79 () {56 vars in 1143 bytes} [Tue Sep 6 21:03:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:55.996198 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa6e2f-6ac5-419e-a520-a0435b2de8e1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:56.110516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa6e2f-6ac5-419e-a520-a0435b2de8e1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:56.147408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fac88-9e30-45c7-8ca8-94e8e1288001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:56.148088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fac88-9e30-45c7-8ca8-94e8e1288001 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:56.148371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fac88-9e30-45c7-8ca8-94e8e1288001 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:56.148644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fac88-9e30-45c7-8ca8-94e8e1288001 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:56.215621 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:56.231612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742fac88-9e30-45c7-8ca8-94e8e1288001 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:56.233468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742fac88-9e30-45c7-8ca8-94e8e1288001 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:56.345463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 629/1255] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:03:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:03:56.346734 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d34d2-fc9d-4c03-969b-b41249c59e45 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:56.461391 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d34d2-fc9d-4c03-969b-b41249c59e45 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:56.476104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:56.499500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6aa76-5990-42dd-b621-a335982e5aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:56.500164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6aa76-5990-42dd-b621-a335982e5aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:56.500432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6aa76-5990-42dd-b621-a335982e5aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:56.500746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6aa76-5990-42dd-b621-a335982e5aa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:56.511928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e6aa76-5990-42dd-b621-a335982e5aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:56.513404 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e6aa76-5990-42dd-b621-a335982e5aa5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e1e6aa76-5990-42dd-b621-a335982e5aa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:03:56.528778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:56.530085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:03:56.531114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:03:56.531114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:03:56.531114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:03:56.531114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 627/1256] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:03:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:56.536158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fac88-9e30-45c7-8ca8-94e8e1288001 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:56.641639 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:56.657851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fac88-9e30-45c7-8ca8-94e8e1288001 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:56.698935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46633a-31ef-4022-a596-8207fe5a0a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:56.699682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46633a-31ef-4022-a596-8207fe5a0a44 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:56.700047 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46633a-31ef-4022-a596-8207fe5a0a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:56.700413 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46633a-31ef-4022-a596-8207fe5a0a44 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:56.701200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46633a-31ef-4022-a596-8207fe5a0a44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:56.701588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46633a-31ef-4022-a596-8207fe5a0a44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:56.703090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46633a-31ef-4022-a596-8207fe5a0a44 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:56.715338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 630/1257] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:56.721167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6aa76-5990-42dd-b621-a335982e5aa5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:56.838948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6aa76-5990-42dd-b621-a335982e5aa5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:56.881580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70eada-e0e1-40d3-bd5b-3d214c0c437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:56.882242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70eada-e0e1-40d3-bd5b-3d214c0c437d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:56.882512 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70eada-e0e1-40d3-bd5b-3d214c0c437d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:56.882786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70eada-e0e1-40d3-bd5b-3d214c0c437d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:56.890438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70eada-e0e1-40d3-bd5b-3d214c0c437d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:56.891934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70eada-e0e1-40d3-bd5b-3d214c0c437d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-3d70eada-e0e1-40d3-bd5b-3d214c0c437d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:03:56.904968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:56.906096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:56.907230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:03:56.907230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:03:56.907230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:03:56.907230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:03:56.907230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:03:56.907230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:03:56.907230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:03:56.907230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:03:56.907230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 628/1258] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:03:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 21:03:56.914453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46633a-31ef-4022-a596-8207fe5a0a44 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:57.026742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46633a-31ef-4022-a596-8207fe5a0a44 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:57.070807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac456e-ba93-40e2-b8da-8b6e1551cc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:57.071432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac456e-ba93-40e2-b8da-8b6e1551cc5b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:57.071705 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac456e-ba93-40e2-b8da-8b6e1551cc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:57.071970 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac456e-ba93-40e2-b8da-8b6e1551cc5b None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:57.072686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac456e-ba93-40e2-b8da-8b6e1551cc5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:57.072979 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac456e-ba93-40e2-b8da-8b6e1551cc5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:57.074467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac456e-ba93-40e2-b8da-8b6e1551cc5b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:57.093693 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 631/1259] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:03:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:57.100087 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70eada-e0e1-40d3-bd5b-3d214c0c437d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:57.142743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:57.234612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70eada-e0e1-40d3-bd5b-3d214c0c437d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:57.273264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc187b25-84a8-4395-9c5c-d80677c7c3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:57.273854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc187b25-84a8-4395-9c5c-d80677c7c3dc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:57.274127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc187b25-84a8-4395-9c5c-d80677c7c3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:57.274394 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc187b25-84a8-4395-9c5c-d80677c7c3dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:57.275129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc187b25-84a8-4395-9c5c-d80677c7c3dc None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=c40b5ff21d1940749bf479cc936fc0f2) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:03:57.275461 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc187b25-84a8-4395-9c5c-d80677c7c3dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:03:57.276924 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc187b25-84a8-4395-9c5c-d80677c7c3dc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:03:57.283567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 629/1260] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:03:57 2022] GET /identity/v3/role_assignments?user.id=c40b5ff21d1940749bf479cc936fc0f2&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=3b30504991194002b41f8587d0851c10 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:03:59.440177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218540d-d440-46de-9074-aa0291d6e3f3 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:59.441081 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218540d-d440-46de-9074-aa0291d6e3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:59.441081 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218540d-d440-46de-9074-aa0291d6e3f3 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:59.443242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 632/1261] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:03:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:03:59.448610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d210869-2529-4bee-9b1e-2b68cedfbc63 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:59.449073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d210869-2529-4bee-9b1e-2b68cedfbc63 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:59.449182 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d210869-2529-4bee-9b1e-2b68cedfbc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:59.469898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9d210869-2529-4bee-9b1e-2b68cedfbc63 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:03:59.471103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9d210869-2529-4bee-9b1e-2b68cedfbc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:03:59.546698 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 630/1262] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:03:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:03:59.808967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218540d-d440-46de-9074-aa0291d6e3f3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:03:59.823442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:03:59.926963 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218540d-d440-46de-9074-aa0291d6e3f3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:03:59.966083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f993ff2-f768-4905-aa92-f6f8e72d6cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:03:59.966711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f993ff2-f768-4905-aa92-f6f8e72d6cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:03:59.966988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f993ff2-f768-4905-aa92-f6f8e72d6cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:03:59.967261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f993ff2-f768-4905-aa92-f6f8e72d6cf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:03:59.978833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f993ff2-f768-4905-aa92-f6f8e72d6cf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4f993ff2-f768-4905-aa92-f6f8e72d6cf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:03:59.983699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:03:59.984856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:03:59.986117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:03:59.986117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:03:59.986117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:03:59.986117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:03:59.986117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:03:59.986117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:03:59.986117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:03:59.986117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:03:59.986117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:03:59.986117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:03:59.986117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:03:59.986117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:03:59.986117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 633/1263] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:03:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:03:59.994582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d210869-2529-4bee-9b1e-2b68cedfbc63 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:00.017665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:00.102822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d210869-2529-4bee-9b1e-2b68cedfbc63 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:00.139714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea6358-8a67-4326-9726-d59aaaf046b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:00.140749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea6358-8a67-4326-9726-d59aaaf046b2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:00.141181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea6358-8a67-4326-9726-d59aaaf046b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:00.141541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea6358-8a67-4326-9726-d59aaaf046b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:00.142643 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea6358-8a67-4326-9726-d59aaaf046b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:00.143085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea6358-8a67-4326-9726-d59aaaf046b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:00.145220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea6358-8a67-4326-9726-d59aaaf046b2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:00.154840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 631/1264] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:03:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:00.161692 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f993ff2-f768-4905-aa92-f6f8e72d6cf8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:00.238923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:00.271884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f993ff2-f768-4905-aa92-f6f8e72d6cf8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:00.315481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9cf6e-4219-4a4c-a89e-fcd26be44206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:00.316114 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9cf6e-4219-4a4c-a89e-fcd26be44206 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:00.316395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9cf6e-4219-4a4c-a89e-fcd26be44206 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:00.316700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9cf6e-4219-4a4c-a89e-fcd26be44206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:00.327964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9cf6e-4219-4a4c-a89e-fcd26be44206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:00.329465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9cf6e-4219-4a4c-a89e-fcd26be44206 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-aab9cf6e-4219-4a4c-a89e-fcd26be44206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:04:00.345339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:04:00.346389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:04:00.347304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:04:00.347304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:04:00.347304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:04:00.347304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 634/1265] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:04:00.353337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea6358-8a67-4326-9726-d59aaaf046b2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:00.448005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:00.478984 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea6358-8a67-4326-9726-d59aaaf046b2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:00.514985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e158504-205b-4daf-be65-5645ee96dbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:00.515589 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158504-205b-4daf-be65-5645ee96dbe3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:00.515870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158504-205b-4daf-be65-5645ee96dbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:00.516146 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158504-205b-4daf-be65-5645ee96dbe3 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:00.516825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e158504-205b-4daf-be65-5645ee96dbe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:00.517128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e158504-205b-4daf-be65-5645ee96dbe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:00.518518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e158504-205b-4daf-be65-5645ee96dbe3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:00.530588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 632/1266] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:00.536899 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9cf6e-4219-4a4c-a89e-fcd26be44206 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:00.642499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9cf6e-4219-4a4c-a89e-fcd26be44206 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:00.664004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:00.681083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ab66-3088-4b91-8eaa-0d136160bb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:00.681709 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ab66-3088-4b91-8eaa-0d136160bb95 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:00.681984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ab66-3088-4b91-8eaa-0d136160bb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:00.682251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ab66-3088-4b91-8eaa-0d136160bb95 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:00.690599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26ab66-3088-4b91-8eaa-0d136160bb95 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:00.692186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26ab66-3088-4b91-8eaa-0d136160bb95 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1f26ab66-3088-4b91-8eaa-0d136160bb95 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:04:00.704409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:04:00.705499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:04:00.706578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:04:00.706578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:04:00.706578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:04:00.706578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:04:00.706578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:04:00.706578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:04:00.706578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:04:00.706578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:04:00.706578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 21:04:00.706578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:04:00.706578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 635/1267] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:04:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:04:00.711677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e158504-205b-4daf-be65-5645ee96dbe3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:00.823135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e158504-205b-4daf-be65-5645ee96dbe3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:00.868639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30492aea-5e63-40bd-92f9-bff56ea2af26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:00.869330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30492aea-5e63-40bd-92f9-bff56ea2af26 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:00.869659 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30492aea-5e63-40bd-92f9-bff56ea2af26 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:00.869996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30492aea-5e63-40bd-92f9-bff56ea2af26 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:00.870816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30492aea-5e63-40bd-92f9-bff56ea2af26 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:00.871226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30492aea-5e63-40bd-92f9-bff56ea2af26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:00.872969 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30492aea-5e63-40bd-92f9-bff56ea2af26 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:00.882484 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:00.891907 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 633/1268] 10.222.0.79 () {56 vars in 1143 bytes} [Tue Sep 6 21:04:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:00.898238 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ab66-3088-4b91-8eaa-0d136160bb95 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:01.005288 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ab66-3088-4b91-8eaa-0d136160bb95 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:01.040615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da25a5e-b021-426c-bf5d-6faca8dfdc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:01.041266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da25a5e-b021-426c-bf5d-6faca8dfdc1b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:01.041540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da25a5e-b021-426c-bf5d-6faca8dfdc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:01.041806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da25a5e-b021-426c-bf5d-6faca8dfdc1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:01.052645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da25a5e-b021-426c-bf5d-6faca8dfdc1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:01.054128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da25a5e-b021-426c-bf5d-6faca8dfdc1b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9da25a5e-b021-426c-bf5d-6faca8dfdc1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:04:01.068913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:04:01.069961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:04:01.070934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:04:01.070934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:04:01.070934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:04:01.070934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 636/1269] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:04:01.076553 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30492aea-5e63-40bd-92f9-bff56ea2af26 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:01.180138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30492aea-5e63-40bd-92f9-bff56ea2af26 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:01.216434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea2097-8d1b-4f80-8267-bee51cb51e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:01.217059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea2097-8d1b-4f80-8267-bee51cb51e33 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:01.217331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea2097-8d1b-4f80-8267-bee51cb51e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:01.217595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea2097-8d1b-4f80-8267-bee51cb51e33 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:01.218227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea2097-8d1b-4f80-8267-bee51cb51e33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:01.218530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea2097-8d1b-4f80-8267-bee51cb51e33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:01.219893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea2097-8d1b-4f80-8267-bee51cb51e33 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:01.231789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 634/1270] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:01.237881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da25a5e-b021-426c-bf5d-6faca8dfdc1b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:01.271166 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:01.341107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da25a5e-b021-426c-bf5d-6faca8dfdc1b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:01.375583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925421e5-ffaf-4363-a2f6-1b3c8a95d597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:01.376226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925421e5-ffaf-4363-a2f6-1b3c8a95d597 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:01.376501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925421e5-ffaf-4363-a2f6-1b3c8a95d597 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:01.376792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925421e5-ffaf-4363-a2f6-1b3c8a95d597 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:01.384149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925421e5-ffaf-4363-a2f6-1b3c8a95d597 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:01.385621 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925421e5-ffaf-4363-a2f6-1b3c8a95d597 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-925421e5-ffaf-4363-a2f6-1b3c8a95d597 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:04:01.403380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:04:01.404861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:04:01.406250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:04:01.406250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:04:01.406250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:04:01.406250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:04:01.406250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:04:01.406250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:04:01.406250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:04:01.406250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:04:01.406250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 637/1271] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:04:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:04:01.413210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea2097-8d1b-4f80-8267-bee51cb51e33 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:01.474255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:01.517506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea2097-8d1b-4f80-8267-bee51cb51e33 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:01.555052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe34eb-e092-43ac-ba7f-ffc609e77bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:01.555656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe34eb-e092-43ac-ba7f-ffc609e77bcf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:01.555932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe34eb-e092-43ac-ba7f-ffc609e77bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:01.556198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe34eb-e092-43ac-ba7f-ffc609e77bcf None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:01.556946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe34eb-e092-43ac-ba7f-ffc609e77bcf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:01.557253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe34eb-e092-43ac-ba7f-ffc609e77bcf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:01.558700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe34eb-e092-43ac-ba7f-ffc609e77bcf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:01.571535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 635/1272] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:04:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:01.578211 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925421e5-ffaf-4363-a2f6-1b3c8a95d597 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:01.684842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:01.694211 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925421e5-ffaf-4363-a2f6-1b3c8a95d597 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:01.729052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba835ea-beb3-4d00-bc40-ae571052798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:01.729632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba835ea-beb3-4d00-bc40-ae571052798a None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:01.729910 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba835ea-beb3-4d00-bc40-ae571052798a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:01.730177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba835ea-beb3-4d00-bc40-ae571052798a None admin] PATH_INFO: `/v3/projects/05af249ba0a047fe83aefb078e203e53/users/c40b5ff21d1940749bf479cc936fc0f2/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:01.732318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba835ea-beb3-4d00-bc40-ae571052798a None admin] RBAC: Authorizing `identity:create_grant(project_id=05af249ba0a047fe83aefb078e203e53, user_id=c40b5ff21d1940749bf479cc936fc0f2, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:01.733796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba835ea-beb3-4d00-bc40-ae571052798a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:01.753939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 638/1273] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Sep 6 21:04:01 2022] PUT /identity/v3/projects/05af249ba0a047fe83aefb078e203e53/users/c40b5ff21d1940749bf479cc936fc0f2/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:04:03.936015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f5a52-b8e8-4ded-a99a-b127b58b0a25 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:03.936867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f5a52-b8e8-4ded-a99a-b127b58b0a25 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:03.936867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f5a52-b8e8-4ded-a99a-b127b58b0a25 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:03.938951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 636/1274] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:04:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:04:03.946914 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03700f1b-2795-48fa-95ef-cb05c8f60a02 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:03.947381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03700f1b-2795-48fa-95ef-cb05c8f60a02 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:03.947472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03700f1b-2795-48fa-95ef-cb05c8f60a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:03.968973 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-03700f1b-2795-48fa-95ef-cb05c8f60a02 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:04:03.970179 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-03700f1b-2795-48fa-95ef-cb05c8f60a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:04:04.048284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 639/1275] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:04:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:04:04.312942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f5a52-b8e8-4ded-a99a-b127b58b0a25 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:04.427828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f5a52-b8e8-4ded-a99a-b127b58b0a25 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:04.447492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:04.476054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0201d-c119-48e2-97ae-d294b796bab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:04.476867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0201d-c119-48e2-97ae-d294b796bab6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:04.477142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0201d-c119-48e2-97ae-d294b796bab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:04.477409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0201d-c119-48e2-97ae-d294b796bab6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:04.488245 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0201d-c119-48e2-97ae-d294b796bab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-cbf0201d-c119-48e2-97ae-d294b796bab6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 21:04:04.493382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:04:04.494364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:04:04.495263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:04:04.495263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:04:04.495263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:04:04.495263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:04:04.495263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 21:04:04.495263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 21:04:04.495263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 21:04:04.495263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 21:04:04.495263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 21:04:04.495263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 21:04:04.495263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 21:04:04.495263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:04:04.495263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 637/1276] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Sep 6 21:04:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:04:04.503559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03700f1b-2795-48fa-95ef-cb05c8f60a02 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:04.590635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:04.611957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03700f1b-2795-48fa-95ef-cb05c8f60a02 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:04.645101 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4e069-39a8-46fd-b2de-f702e8aa992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:04.645698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4e069-39a8-46fd-b2de-f702e8aa992b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:04.645969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4e069-39a8-46fd-b2de-f702e8aa992b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:04.646238 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4e069-39a8-46fd-b2de-f702e8aa992b None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:04.646893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4e069-39a8-46fd-b2de-f702e8aa992b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:04.647191 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4e069-39a8-46fd-b2de-f702e8aa992b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:04.648579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4e069-39a8-46fd-b2de-f702e8aa992b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:04.657363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 640/1277] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Sep 6 21:04:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:04.664212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0201d-c119-48e2-97ae-d294b796bab6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:04.781091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0201d-c119-48e2-97ae-d294b796bab6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:04.823399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b17ba1-3a2a-499f-a7fb-a10dd1c23c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:04.824273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b17ba1-3a2a-499f-a7fb-a10dd1c23c97 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:04.824757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b17ba1-3a2a-499f-a7fb-a10dd1c23c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:04.825183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b17ba1-3a2a-499f-a7fb-a10dd1c23c97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:04.841014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b17ba1-3a2a-499f-a7fb-a10dd1c23c97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:04.842750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b17ba1-3a2a-499f-a7fb-a10dd1c23c97 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e2b17ba1-3a2a-499f-a7fb-a10dd1c23c97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:04:04.860567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:04:04.861677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:04:04.862667 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:04:04.862667 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:04:04.862667 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:04:04.862667 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 638/1278] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:04:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:04:04.868734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4e069-39a8-46fd-b2de-f702e8aa992b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:04.979816 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4e069-39a8-46fd-b2de-f702e8aa992b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:05.021633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c23ccd-02be-4c85-a8dc-0c2b93f7dfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:05.022235 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c23ccd-02be-4c85-a8dc-0c2b93f7dfcc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:05.022506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c23ccd-02be-4c85-a8dc-0c2b93f7dfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:05.022768 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c23ccd-02be-4c85-a8dc-0c2b93f7dfcc None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:05.023410 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c23ccd-02be-4c85-a8dc-0c2b93f7dfcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:05.023702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c23ccd-02be-4c85-a8dc-0c2b93f7dfcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:05.025105 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c23ccd-02be-4c85-a8dc-0c2b93f7dfcc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:05.036448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 641/1279] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:05.043320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b17ba1-3a2a-499f-a7fb-a10dd1c23c97 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:05.087103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:05.169609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b17ba1-3a2a-499f-a7fb-a10dd1c23c97 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:05.206573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bba28-b494-4168-a6a3-215fb559f196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:05.207241 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bba28-b494-4168-a6a3-215fb559f196 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:05.207520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bba28-b494-4168-a6a3-215fb559f196 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:05.207790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bba28-b494-4168-a6a3-215fb559f196 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:05.215527 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bba28-b494-4168-a6a3-215fb559f196 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:05.216996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bba28-b494-4168-a6a3-215fb559f196 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-679bba28-b494-4168-a6a3-215fb559f196 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 21:04:05.228438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:04:05.229547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:04:05.230713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:04:05.230713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:04:05.230713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 21:04:05.230713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 21:04:05.230713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 21:04:05.230713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 21:04:05.230713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 21:04:05.230713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 21:04:05.230713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 21:04:05.230713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:04:05.233000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 639/1280] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:04:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:04:05.235506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c23ccd-02be-4c85-a8dc-0c2b93f7dfcc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:05.239643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:05.343871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c23ccd-02be-4c85-a8dc-0c2b93f7dfcc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:05.381146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1853c0-a188-442d-ac7f-00cea95e9720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:05.381748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1853c0-a188-442d-ac7f-00cea95e9720 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:05.382023 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1853c0-a188-442d-ac7f-00cea95e9720 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:05.382292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1853c0-a188-442d-ac7f-00cea95e9720 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:05.383017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1853c0-a188-442d-ac7f-00cea95e9720 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:05.383315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1853c0-a188-442d-ac7f-00cea95e9720 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:05.384789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1853c0-a188-442d-ac7f-00cea95e9720 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:05.401644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 642/1281] 10.222.0.79 () {56 vars in 1143 bytes} [Tue Sep 6 21:04:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:05.407811 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bba28-b494-4168-a6a3-215fb559f196 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:05.507801 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:05.519912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bba28-b494-4168-a6a3-215fb559f196 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:05.553592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6cf15-7432-4093-8783-42be21e372e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:05.554190 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6cf15-7432-4093-8783-42be21e372e4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:05.554466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6cf15-7432-4093-8783-42be21e372e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:05.554923 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6cf15-7432-4093-8783-42be21e372e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:05.636838 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6cf15-7432-4093-8783-42be21e372e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:05.638307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6cf15-7432-4093-8783-42be21e372e4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:05.743107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:05.754507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 640/1282] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Sep 6 21:04:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 21:04:05.760831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1853c0-a188-442d-ac7f-00cea95e9720 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:05.827898 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:05.866514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1853c0-a188-442d-ac7f-00cea95e9720 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:05.910611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b2318-061d-446c-a79c-b51d9e46b9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:05.911247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b2318-061d-446c-a79c-b51d9e46b9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:05.911520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b2318-061d-446c-a79c-b51d9e46b9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:05.911788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b2318-061d-446c-a79c-b51d9e46b9a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:05.922828 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b2318-061d-446c-a79c-b51d9e46b9a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:05.924279 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b2318-061d-446c-a79c-b51d9e46b9a3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2e3b2318-061d-446c-a79c-b51d9e46b9a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 21:04:05.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:04:05.941739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 21:04:05.942823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 21:04:05.942823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 21:04:05.942823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:04:05.942823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 643/1283] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:04:05.947718 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6cf15-7432-4093-8783-42be21e372e4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:06.062713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6cf15-7432-4093-8783-42be21e372e4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:06.098992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902c50b-6abd-46d5-8dd5-6168e5dfa18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:06.098992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902c50b-6abd-46d5-8dd5-6168e5dfa18c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:06.098992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902c50b-6abd-46d5-8dd5-6168e5dfa18c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:06.098992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902c50b-6abd-46d5-8dd5-6168e5dfa18c None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:06.099551 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902c50b-6abd-46d5-8dd5-6168e5dfa18c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:06.099852 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902c50b-6abd-46d5-8dd5-6168e5dfa18c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:06.101256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902c50b-6abd-46d5-8dd5-6168e5dfa18c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:06.112864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 641/1284] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Sep 6 21:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:06.118927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b2318-061d-446c-a79c-b51d9e46b9a3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:06.234712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b2318-061d-446c-a79c-b51d9e46b9a3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:06.249277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:06.277472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b9ec4-e016-43bf-accf-3b093d1c14f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:06.278112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b9ec4-e016-43bf-accf-3b093d1c14f4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:06.278385 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b9ec4-e016-43bf-accf-3b093d1c14f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:06.278653 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b9ec4-e016-43bf-accf-3b093d1c14f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:06.285671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b9ec4-e016-43bf-accf-3b093d1c14f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:06.287127 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b9ec4-e016-43bf-accf-3b093d1c14f4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b67b9ec4-e016-43bf-accf-3b093d1c14f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 21:04:06.298812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:04:06.299824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:04:06.301253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 21:04:06.301253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 21:04:06.301253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 21:04:06.301253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 21:04:06.301253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 21:04:06.301253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 21:04:06.301253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 21:04:06.301253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 21:04:06.301253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 644/1285] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Sep 6 21:04:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:04:06.308080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902c50b-6abd-46d5-8dd5-6168e5dfa18c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:06.364348 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:06.417927 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902c50b-6abd-46d5-8dd5-6168e5dfa18c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:06.456143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fbf16-3481-4522-bdbc-898fc7b3c802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:06.456776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fbf16-3481-4522-bdbc-898fc7b3c802 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:06.457057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fbf16-3481-4522-bdbc-898fc7b3c802 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:06.457352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fbf16-3481-4522-bdbc-898fc7b3c802 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:06.458042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861fbf16-3481-4522-bdbc-898fc7b3c802 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:06.458345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861fbf16-3481-4522-bdbc-898fc7b3c802 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:06.459790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861fbf16-3481-4522-bdbc-898fc7b3c802 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:06.471973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 642/1286] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Sep 6 21:04:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:06.481934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b9ec4-e016-43bf-accf-3b093d1c14f4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:06.597414 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b9ec4-e016-43bf-accf-3b093d1c14f4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:06.633011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e405b-c48c-4acb-92f0-299f5775312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:06.633538 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e405b-c48c-4acb-92f0-299f5775312f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:06.633818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e405b-c48c-4acb-92f0-299f5775312f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:06.634086 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e405b-c48c-4acb-92f0-299f5775312f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:06.634816 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e405b-c48c-4acb-92f0-299f5775312f None admin] RBAC: Adding query filter params (role.id=3b30504991194002b41f8587d0851c10, scope.project.id=05af249ba0a047fe83aefb078e203e53, user.id=c40b5ff21d1940749bf479cc936fc0f2) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:06.635124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e405b-c48c-4acb-92f0-299f5775312f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:06.636579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e405b-c48c-4acb-92f0-299f5775312f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:06.642505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 645/1287] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Sep 6 21:04:06 2022] GET /identity/v3/role_assignments?user.id=c40b5ff21d1940749bf479cc936fc0f2&scope.project.id=05af249ba0a047fe83aefb078e203e53&role.id=3b30504991194002b41f8587d0851c10 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:08.781276 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671f26b-c5ee-4535-80c6-21cba9015918 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:08.782180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671f26b-c5ee-4535-80c6-21cba9015918 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:08.782180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671f26b-c5ee-4535-80c6-21cba9015918 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:08.784336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 643/1288] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:04:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:04:08.789901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa051dd-f68c-4822-90ad-e1b4e9ed328a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:08.790348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa051dd-f68c-4822-90ad-e1b4e9ed328a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:08.790436 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa051dd-f68c-4822-90ad-e1b4e9ed328a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:08.810778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1aa051dd-f68c-4822-90ad-e1b4e9ed328a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:04:08.812015 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1aa051dd-f68c-4822-90ad-e1b4e9ed328a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:04:08.848320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:08.893818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 646/1289] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:04:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:04:09.159744 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671f26b-c5ee-4535-80c6-21cba9015918 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:09.277181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671f26b-c5ee-4535-80c6-21cba9015918 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:09.287967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:09.319175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d73c42-646c-4ba4-8713-125a9530940a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:09.319795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d73c42-646c-4ba4-8713-125a9530940a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:09.320061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d73c42-646c-4ba4-8713-125a9530940a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:09.320446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d73c42-646c-4ba4-8713-125a9530940a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:09.321592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d73c42-646c-4ba4-8713-125a9530940a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:09.323833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d73c42-646c-4ba4-8713-125a9530940a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b7d73c42-646c-4ba4-8713-125a9530940a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:04:09.333456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:04:09.336874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:04:09.337965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 644/1290] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Sep 6 21:04:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:04:09.341047 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa051dd-f68c-4822-90ad-e1b4e9ed328a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:09.448904 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa051dd-f68c-4822-90ad-e1b4e9ed328a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:09.484521 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697fe3e-db16-4243-b369-82744dd702e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:09.485156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697fe3e-db16-4243-b369-82744dd702e6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:09.485429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697fe3e-db16-4243-b369-82744dd702e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:09.485701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697fe3e-db16-4243-b369-82744dd702e6 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:09.486332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697fe3e-db16-4243-b369-82744dd702e6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:09.486631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697fe3e-db16-4243-b369-82744dd702e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:09.487999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697fe3e-db16-4243-b369-82744dd702e6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:09.493419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 647/1291] 10.222.0.79 () {56 vars in 1116 bytes} [Tue Sep 6 21:04:09 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:09.502781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d73c42-646c-4ba4-8713-125a9530940a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:09.611070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d73c42-646c-4ba4-8713-125a9530940a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:09.648161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ffdc0-5ef9-4e77-8c4c-59c1424950b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:09.648793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffdc0-5ef9-4e77-8c4c-59c1424950b5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:09.649067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffdc0-5ef9-4e77-8c4c-59c1424950b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:09.649336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffdc0-5ef9-4e77-8c4c-59c1424950b5 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:09.649932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ffdc0-5ef9-4e77-8c4c-59c1424950b5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:09.650225 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ffdc0-5ef9-4e77-8c4c-59c1424950b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:09.651633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ffdc0-5ef9-4e77-8c4c-59c1424950b5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:09.656961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 645/1292] 10.222.0.79 () {56 vars in 1116 bytes} [Tue Sep 6 21:04:09 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:11.790942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a29c9-deb3-409b-9bd8-f54f14a02682 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:11.791799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a29c9-deb3-409b-9bd8-f54f14a02682 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:11.791799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a29c9-deb3-409b-9bd8-f54f14a02682 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:11.793992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 648/1293] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:04:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:04:11.800321 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424201b-cad8-4919-a84b-d85f398b9bf0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:11.800796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424201b-cad8-4919-a84b-d85f398b9bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:11.800915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424201b-cad8-4919-a84b-d85f398b9bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:11.826096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0424201b-cad8-4919-a84b-d85f398b9bf0 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:04:11.827316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0424201b-cad8-4919-a84b-d85f398b9bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:04:11.861088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:11.908602 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 646/1294] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:04:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:04:12.171218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a29c9-deb3-409b-9bd8-f54f14a02682 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:12.216948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:12.303772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a29c9-deb3-409b-9bd8-f54f14a02682 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:12.340670 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29188f0b-15f6-412b-90f8-79c1545e5213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:12.341300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188f0b-15f6-412b-90f8-79c1545e5213 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:12.341575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188f0b-15f6-412b-90f8-79c1545e5213 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:12.341845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188f0b-15f6-412b-90f8-79c1545e5213 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:12.342495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29188f0b-15f6-412b-90f8-79c1545e5213 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:12.343957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29188f0b-15f6-412b-90f8-79c1545e5213 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:12.361727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 649/1295] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Sep 6 21:04:12 2022] POST /identity/v3/services => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:04:14.539114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37909d3a-8141-4a1b-b789-4f85d4767ec8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:14.540069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37909d3a-8141-4a1b-b789-4f85d4767ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:14.540069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37909d3a-8141-4a1b-b789-4f85d4767ec8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:14.542300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 647/1296] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:04:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:04:14.551169 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f0e4d-0f07-4eec-9a14-2063147c9e53 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:14.551620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f0e4d-0f07-4eec-9a14-2063147c9e53 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:14.551722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f0e4d-0f07-4eec-9a14-2063147c9e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:14.573770 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fb0f0e4d-0f07-4eec-9a14-2063147c9e53 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:04:14.574967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fb0f0e4d-0f07-4eec-9a14-2063147c9e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:04:14.656240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 650/1297] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:04:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:04:14.919856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37909d3a-8141-4a1b-b789-4f85d4767ec8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:15.038364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37909d3a-8141-4a1b-b789-4f85d4767ec8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:15.074403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68e68d-ded1-41e1-b88b-e216c8628026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:15.075020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68e68d-ded1-41e1-b88b-e216c8628026 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:15.075296 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68e68d-ded1-41e1-b88b-e216c8628026 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:15.075566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68e68d-ded1-41e1-b88b-e216c8628026 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:15.076151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68e68d-ded1-41e1-b88b-e216c8628026 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:15.077560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68e68d-ded1-41e1-b88b-e216c8628026 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2d68e68d-ded1-41e1-b88b-e216c8628026 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:04:15.087154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:04:15.088364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:04:15.089626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 648/1298] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Sep 6 21:04:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:04:15.096344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f0e4d-0f07-4eec-9a14-2063147c9e53 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:15.122147 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:15.207148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f0e4d-0f07-4eec-9a14-2063147c9e53 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:15.242732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229415d-d054-4851-82c9-fd7b169575f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:15.243339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229415d-d054-4851-82c9-fd7b169575f0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:15.243614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229415d-d054-4851-82c9-fd7b169575f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:15.243884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229415d-d054-4851-82c9-fd7b169575f0 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:15.244509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229415d-d054-4851-82c9-fd7b169575f0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:15.244826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229415d-d054-4851-82c9-fd7b169575f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:15.246217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229415d-d054-4851-82c9-fd7b169575f0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:15.251612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 651/1299] 10.222.0.79 () {56 vars in 1116 bytes} [Tue Sep 6 21:04:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:15.257987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68e68d-ded1-41e1-b88b-e216c8628026 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:15.291255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:15.361181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68e68d-ded1-41e1-b88b-e216c8628026 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:15.395198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9c2eb-97e1-4368-b005-a70b53df2649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:15.395794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9c2eb-97e1-4368-b005-a70b53df2649 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:15.396067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9c2eb-97e1-4368-b005-a70b53df2649 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:15.396335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9c2eb-97e1-4368-b005-a70b53df2649 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:15.397068 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9c2eb-97e1-4368-b005-a70b53df2649 None admin] RBAC: Adding query filter params (interface=public, service_id=82df3340423847418b89b01fce676feb, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:15.397371 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9c2eb-97e1-4368-b005-a70b53df2649 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:15.398733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9c2eb-97e1-4368-b005-a70b53df2649 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:15.404345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 649/1300] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Sep 6 21:04:15 2022] GET /identity/v3/endpoints?service_id=82df3340423847418b89b01fce676feb&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:15.412941 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229415d-d054-4851-82c9-fd7b169575f0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:15.496633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:15.515874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229415d-d054-4851-82c9-fd7b169575f0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:15.549568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459aeab1-f410-4fb4-8096-3ae308f149a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:15.550171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459aeab1-f410-4fb4-8096-3ae308f149a6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:15.550448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459aeab1-f410-4fb4-8096-3ae308f149a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:15.550714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459aeab1-f410-4fb4-8096-3ae308f149a6 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:15.551338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459aeab1-f410-4fb4-8096-3ae308f149a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:15.552746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459aeab1-f410-4fb4-8096-3ae308f149a6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:15.560087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 652/1301] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Sep 6 21:04:15 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:04:17.693419 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09913-68a0-4b0f-bd10-cf99af780034 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:17.694004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09913-68a0-4b0f-bd10-cf99af780034 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:17.694004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09913-68a0-4b0f-bd10-cf99af780034 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:17.695736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 650/1302] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:04:17.702605 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315e41-4e44-4a69-bdd5-4211c9bd0ebc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:17.703072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315e41-4e44-4a69-bdd5-4211c9bd0ebc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:17.703171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315e41-4e44-4a69-bdd5-4211c9bd0ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:17.727463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-22315e41-4e44-4a69-bdd5-4211c9bd0ebc None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:04:17.728691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-22315e41-4e44-4a69-bdd5-4211c9bd0ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:04:17.803333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 653/1303] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:04:17 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 21:04:18.106411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e09913-68a0-4b0f-bd10-cf99af780034 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:18.214381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:18.221983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e09913-68a0-4b0f-bd10-cf99af780034 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:18.256014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a91fa-2e11-4355-9733-9fe4e9717075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:18.256623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a91fa-2e11-4355-9733-9fe4e9717075 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:18.256925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a91fa-2e11-4355-9733-9fe4e9717075 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:18.257196 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a91fa-2e11-4355-9733-9fe4e9717075 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:18.257884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a91fa-2e11-4355-9733-9fe4e9717075 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:18.259133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a91fa-2e11-4355-9733-9fe4e9717075 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-326a91fa-2e11-4355-9733-9fe4e9717075 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 21:04:18.267930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 21:04:18.269179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 21:04:18.270229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 651/1304] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Sep 6 21:04:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 21:04:18.276003 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22315e41-4e44-4a69-bdd5-4211c9bd0ebc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:18.365142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:18.389038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22315e41-4e44-4a69-bdd5-4211c9bd0ebc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:18.427217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35007296-e357-4754-a1a1-bd11b6abfcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:18.427849 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35007296-e357-4754-a1a1-bd11b6abfcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:18.428121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35007296-e357-4754-a1a1-bd11b6abfcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:18.428383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35007296-e357-4754-a1a1-bd11b6abfcd8 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:18.429111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35007296-e357-4754-a1a1-bd11b6abfcd8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:04:18.429554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35007296-e357-4754-a1a1-bd11b6abfcd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:18.431760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35007296-e357-4754-a1a1-bd11b6abfcd8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:18.441627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 654/1305] 10.222.0.79 () {56 vars in 1116 bytes} [Tue Sep 6 21:04:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:04:18.448781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a91fa-2e11-4355-9733-9fe4e9717075 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:04:18.563394 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a91fa-2e11-4355-9733-9fe4e9717075 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:04:18.601252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e27a5-98b8-4e5b-a25d-6dcbe3e4cdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:04:18.601886 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e27a5-98b8-4e5b-a25d-6dcbe3e4cdcf None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:04:18.602158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e27a5-98b8-4e5b-a25d-6dcbe3e4cdcf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:04:18.602421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e27a5-98b8-4e5b-a25d-6dcbe3e4cdcf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:04:18.603078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e27a5-98b8-4e5b-a25d-6dcbe3e4cdcf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:04:18.604444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e27a5-98b8-4e5b-a25d-6dcbe3e4cdcf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:04:18.626974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 652/1306] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:04:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:04:28.435673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=30 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:04:28.622143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:24.056368 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2c342-0e59-46e5-9df5-0d077f665523 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:24.057495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2c342-0e59-46e5-9df5-0d077f665523 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:24.058292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2c342-0e59-46e5-9df5-0d077f665523 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:24.062259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 655/1307] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:05:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:05:24.067331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9ca8e-6b05-4b64-8ced-f1cd9eaa6574 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:24.067757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9ca8e-6b05-4b64-8ced-f1cd9eaa6574 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:24.067850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9ca8e-6b05-4b64-8ced-f1cd9eaa6574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:24.164722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-99c9ca8e-6b05-4b64-8ced-f1cd9eaa6574 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:05:24.166292 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-99c9ca8e-6b05-4b64-8ced-f1cd9eaa6574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:24.284891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 653/1308] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:05:24.574338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf63bd-2799-44f0-9c6b-51335bbdcadc None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:24.574598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf63bd-2799-44f0-9c6b-51335bbdcadc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:24.574866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf63bd-2799-44f0-9c6b-51335bbdcadc None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:24.576211 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 656/1309] 10.222.0.79 () {54 vars in 965 bytes} [Tue Sep 6 21:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:05:24.586640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84c6af-52a3-440e-974a-121abb09d5eb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:24.586640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84c6af-52a3-440e-974a-121abb09d5eb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:24.586640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84c6af-52a3-440e-974a-121abb09d5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:24.670805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3b84c6af-52a3-440e-974a-121abb09d5eb None None] MFA Rules not processed for user `d5a45dec40844877a6b5b301fc44a110`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:05:24.672059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3b84c6af-52a3-440e-974a-121abb09d5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:24.703011 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:24.772422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 654/1310] 10.222.0.79 () {58 vars in 1062 bytes} [Tue Sep 6 21:05:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:05:24.781692 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf63bd-2799-44f0-9c6b-51335bbdcadc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:05:24.783771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9dcf63bd-2799-44f0-9c6b-51335bbdcadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:24.905378 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf63bd-2799-44f0-9c6b-51335bbdcadc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:05:24.995396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4050ca-5d3d-4c56-98e2-bfe9511246d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:05:24.995396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4050ca-5d3d-4c56-98e2-bfe9511246d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:24.995396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4050ca-5d3d-4c56-98e2-bfe9511246d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:24.995396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4050ca-5d3d-4c56-98e2-bfe9511246d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:24.995870 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4c4050ca-5d3d-4c56-98e2-bfe9511246d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:25.065187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:25.129355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4050ca-5d3d-4c56-98e2-bfe9511246d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:05:25.131214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4050ca-5d3d-4c56-98e2-bfe9511246d0 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:05:25.151725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 657/1311] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:05:27.600527 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48166c3b-265b-4180-874b-bef0ca32b2a1 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:27.601061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48166c3b-265b-4180-874b-bef0ca32b2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:27.601061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48166c3b-265b-4180-874b-bef0ca32b2a1 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:27.602599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 655/1312] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:05:27.608276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586be7a8-64f1-4fb7-b66e-765740b9d028 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:27.608730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586be7a8-64f1-4fb7-b66e-765740b9d028 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:27.608971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586be7a8-64f1-4fb7-b66e-765740b9d028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:27.681431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-586be7a8-64f1-4fb7-b66e-765740b9d028 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:05:27.682649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-586be7a8-64f1-4fb7-b66e-765740b9d028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:27.785401 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 658/1313] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:05:28.072343 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48166c3b-265b-4180-874b-bef0ca32b2a1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:05:28.073958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-48166c3b-265b-4180-874b-bef0ca32b2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:28.185353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48166c3b-265b-4180-874b-bef0ca32b2a1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:05:28.260278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:28.263299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51671349-9d6e-477f-bc99-3f066bea8ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:05:28.264022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51671349-9d6e-477f-bc99-3f066bea8ccf service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:28.264227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51671349-9d6e-477f-bc99-3f066bea8ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:28.264500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51671349-9d6e-477f-bc99-3f066bea8ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:28.266812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-51671349-9d6e-477f-bc99-3f066bea8ccf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:28.380327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51671349-9d6e-477f-bc99-3f066bea8ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:05:28.381900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51671349-9d6e-477f-bc99-3f066bea8ccf service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:05:28.403739 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 656/1314] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:05:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:05:30.864095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c58bb-8eba-44dd-bddf-ebd6d888b369 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:30.864633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c58bb-8eba-44dd-bddf-ebd6d888b369 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:30.864633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c58bb-8eba-44dd-bddf-ebd6d888b369 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:30.866599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 659/1315] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:05:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:05:30.876262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afe93c-a843-4a4c-a103-f379ac98a92a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:30.876770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afe93c-a843-4a4c-a103-f379ac98a92a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:30.876864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afe93c-a843-4a4c-a103-f379ac98a92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:30.966998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-30afe93c-a843-4a4c-a103-f379ac98a92a None None] MFA Rules not processed for user `40fbba8ecae846bca4e3b0fd6eaf8c85`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:05:30.968268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-30afe93c-a843-4a4c-a103-f379ac98a92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:31.028463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:31.075847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 657/1316] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:05:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:05:31.373529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c58bb-8eba-44dd-bddf-ebd6d888b369 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:05:31.375077 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-619c58bb-8eba-44dd-bddf-ebd6d888b369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:31.385887 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:31.490439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c58bb-8eba-44dd-bddf-ebd6d888b369 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:05:31.589060 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5843d-35d8-4749-8ed1-2839fbcc71ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:05:31.589893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5843d-35d8-4749-8ed1-2839fbcc71ff service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:31.590311 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5843d-35d8-4749-8ed1-2839fbcc71ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:31.590668 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5843d-35d8-4749-8ed1-2839fbcc71ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:31.592903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-61d5843d-35d8-4749-8ed1-2839fbcc71ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:31.663582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:31.714525 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5843d-35d8-4749-8ed1-2839fbcc71ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:05:31.715993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5843d-35d8-4749-8ed1-2839fbcc71ff service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:05:31.735554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 660/1317] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:05:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:05:34.449868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce98d0e-09c0-40c6-82be-1b1bc5de073b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:34.450432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce98d0e-09c0-40c6-82be-1b1bc5de073b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:34.450432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce98d0e-09c0-40c6-82be-1b1bc5de073b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:34.452834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 658/1318] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:05:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:05:34.461116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f964947-9f35-43d5-a8a4-3f274357ba60 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:34.461993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f964947-9f35-43d5-a8a4-3f274357ba60 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:34.462209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f964947-9f35-43d5-a8a4-3f274357ba60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:34.537927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7f964947-9f35-43d5-a8a4-3f274357ba60 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:05:34.539141 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7f964947-9f35-43d5-a8a4-3f274357ba60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:34.646968 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 661/1319] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:05:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:05:34.950628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0f054-bf94-4f3f-b9b6-fb70cfd6f6f6 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:34.950860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0f054-bf94-4f3f-b9b6-fb70cfd6f6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:34.951161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0f054-bf94-4f3f-b9b6-fb70cfd6f6f6 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:34.952531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 659/1320] 10.222.0.79 () {54 vars in 965 bytes} [Tue Sep 6 21:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:05:34.958721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e805b3-75ac-488a-958b-91ac70fea734 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:34.959021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e805b3-75ac-488a-958b-91ac70fea734 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:34.959292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e805b3-75ac-488a-958b-91ac70fea734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:34.984995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:35.056857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-32e805b3-75ac-488a-958b-91ac70fea734 None None] MFA Rules not processed for user `d5a45dec40844877a6b5b301fc44a110`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:05:35.058948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-32e805b3-75ac-488a-958b-91ac70fea734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:35.143183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 662/1321] 10.222.0.79 () {58 vars in 1062 bytes} [Tue Sep 6 21:05:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:05:35.152374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0f054-bf94-4f3f-b9b6-fb70cfd6f6f6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:05:35.153960 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e3c0f054-bf94-4f3f-b9b6-fb70cfd6f6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:35.260920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0f054-bf94-4f3f-b9b6-fb70cfd6f6f6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:05:35.360367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5056e3-77d8-4717-a713-2b86ea9012ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:05:35.361000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5056e3-77d8-4717-a713-2b86ea9012ca service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:35.361274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5056e3-77d8-4717-a713-2b86ea9012ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:35.361537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5056e3-77d8-4717-a713-2b86ea9012ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:35.363147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2b5056e3-77d8-4717-a713-2b86ea9012ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:35.371134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:35.502444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5056e3-77d8-4717-a713-2b86ea9012ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:05:35.503916 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5056e3-77d8-4717-a713-2b86ea9012ca service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:05:35.525671 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 660/1322] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:05:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:05:38.818288 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6522e-d65b-46f3-a348-fd2be8cd9138 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:38.818845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6522e-d65b-46f3-a348-fd2be8cd9138 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:38.818845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6522e-d65b-46f3-a348-fd2be8cd9138 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:38.820156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 663/1323] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:05:38.827357 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f54aa7-a84e-41eb-8f4c-f102aada0a06 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:38.827848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f54aa7-a84e-41eb-8f4c-f102aada0a06 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:38.828190 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f54aa7-a84e-41eb-8f4c-f102aada0a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:38.899393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-52f54aa7-a84e-41eb-8f4c-f102aada0a06 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:05:38.900530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-52f54aa7-a84e-41eb-8f4c-f102aada0a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:38.966142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:39.009961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 661/1324] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:05:39.283041 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6522e-d65b-46f3-a348-fd2be8cd9138 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:05:39.284695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0fe6522e-d65b-46f3-a348-fd2be8cd9138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:39.395406 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6522e-d65b-46f3-a348-fd2be8cd9138 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:05:39.409782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:39.473745 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55944cda-3727-465d-845f-590d07325bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:05:39.474354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55944cda-3727-465d-845f-590d07325bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:39.474625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55944cda-3727-465d-845f-590d07325bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:39.474889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55944cda-3727-465d-845f-590d07325bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:39.476491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-55944cda-3727-465d-845f-590d07325bb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:39.591829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55944cda-3727-465d-845f-590d07325bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:05:39.593319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55944cda-3727-465d-845f-590d07325bb1 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:05:39.611689 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 664/1325] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:05:41.826399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb67bc-a306-4569-bec5-0c44398ccbcb None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:41.826962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb67bc-a306-4569-bec5-0c44398ccbcb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:41.826962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb67bc-a306-4569-bec5-0c44398ccbcb None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:41.828466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 662/1326] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:05:41.834044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8afcdd-282f-4cde-a7fa-25f909d05d2d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:41.834496 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8afcdd-282f-4cde-a7fa-25f909d05d2d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:41.834710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8afcdd-282f-4cde-a7fa-25f909d05d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:41.883976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:41.902592 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9f8afcdd-282f-4cde-a7fa-25f909d05d2d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:05:41.903806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9f8afcdd-282f-4cde-a7fa-25f909d05d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:42.016149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 665/1327] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:05:42.297496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb67bc-a306-4569-bec5-0c44398ccbcb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:05:42.299059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6aeb67bc-a306-4569-bec5-0c44398ccbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:42.407208 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb67bc-a306-4569-bec5-0c44398ccbcb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:05:42.489935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d40863-2057-44dd-af46-f23b595d3486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:05:42.490555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d40863-2057-44dd-af46-f23b595d3486 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:42.490830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d40863-2057-44dd-af46-f23b595d3486 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:42.491100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d40863-2057-44dd-af46-f23b595d3486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:42.493167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-48d40863-2057-44dd-af46-f23b595d3486 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:42.511210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:42.613452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d40863-2057-44dd-af46-f23b595d3486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:05:42.614950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d40863-2057-44dd-af46-f23b595d3486 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:05:42.634809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 663/1328] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:05:44.845362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801203b8-e179-46cf-ae7d-23b37d88526e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:44.845952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801203b8-e179-46cf-ae7d-23b37d88526e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:44.845952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801203b8-e179-46cf-ae7d-23b37d88526e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:44.847235 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 666/1329] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:05:44.852911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef96b2-aade-4292-bdb7-3014b74ab2bd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:44.853399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef96b2-aade-4292-bdb7-3014b74ab2bd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:44.853679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef96b2-aade-4292-bdb7-3014b74ab2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:44.929409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4eef96b2-aade-4292-bdb7-3014b74ab2bd None None] MFA Rules not processed for user `40fbba8ecae846bca4e3b0fd6eaf8c85`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:05:44.930799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4eef96b2-aade-4292-bdb7-3014b74ab2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:45.004123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:45.037043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 664/1330] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:05:44 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:05:45.322455 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801203b8-e179-46cf-ae7d-23b37d88526e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:05:45.324168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-801203b8-e179-46cf-ae7d-23b37d88526e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:45.441696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801203b8-e179-46cf-ae7d-23b37d88526e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:05:45.465603 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:45.526249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd162df-e38c-462c-a577-ead33e6c808b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:05:45.527181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd162df-e38c-462c-a577-ead33e6c808b service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:45.527740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd162df-e38c-462c-a577-ead33e6c808b service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:45.528166 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd162df-e38c-462c-a577-ead33e6c808b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:45.530015 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8bd162df-e38c-462c-a577-ead33e6c808b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:45.668925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd162df-e38c-462c-a577-ead33e6c808b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:05:45.670392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd162df-e38c-462c-a577-ead33e6c808b service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:05:45.688811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 667/1331] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:05:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:05:48.069425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba76eb9-9378-49e4-8fb0-825514cfdc74 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:48.070035 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba76eb9-9378-49e4-8fb0-825514cfdc74 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:48.070035 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba76eb9-9378-49e4-8fb0-825514cfdc74 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:48.071398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 665/1332] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:05:48.079419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d514260-086e-4544-a59e-b4b6a06ff2e1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:48.079419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d514260-086e-4544-a59e-b4b6a06ff2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:48.079419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d514260-086e-4544-a59e-b4b6a06ff2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:48.144137 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:48.146789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2d514260-086e-4544-a59e-b4b6a06ff2e1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:05:48.148505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2d514260-086e-4544-a59e-b4b6a06ff2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:48.256884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 668/1333] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:05:48.541066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba76eb9-9378-49e4-8fb0-825514cfdc74 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:05:48.543490 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8ba76eb9-9378-49e4-8fb0-825514cfdc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:48.668066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba76eb9-9378-49e4-8fb0-825514cfdc74 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:05:48.746110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a9b16-2024-41f8-9213-f7c22dbb362c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:05:48.746733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a9b16-2024-41f8-9213-f7c22dbb362c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:48.747004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a9b16-2024-41f8-9213-f7c22dbb362c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:48.747262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a9b16-2024-41f8-9213-f7c22dbb362c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:48.748924 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-322a9b16-2024-41f8-9213-f7c22dbb362c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:48.776939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:48.878247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a9b16-2024-41f8-9213-f7c22dbb362c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:05:48.879623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a9b16-2024-41f8-9213-f7c22dbb362c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:05:48.898586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 666/1334] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:05:51.103322 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938aed0-83ec-4eb9-a77b-c12b334f220d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:51.104222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938aed0-83ec-4eb9-a77b-c12b334f220d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:51.104222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938aed0-83ec-4eb9-a77b-c12b334f220d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:51.105734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 669/1335] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:05:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:05:51.111705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de774f-f760-42aa-be39-e8f23b9d9783 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:51.112137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de774f-f760-42aa-be39-e8f23b9d9783 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:51.112247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de774f-f760-42aa-be39-e8f23b9d9783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:51.183983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c3de774f-f760-42aa-be39-e8f23b9d9783 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:05:51.185625 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c3de774f-f760-42aa-be39-e8f23b9d9783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:51.262252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:51.285015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 667/1336] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:05:51.566143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938aed0-83ec-4eb9-a77b-c12b334f220d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:05:51.567748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7938aed0-83ec-4eb9-a77b-c12b334f220d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:51.678665 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938aed0-83ec-4eb9-a77b-c12b334f220d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:05:51.709232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:51.756587 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf3177-ddf0-464a-b639-601f466dc787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:05:51.757225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf3177-ddf0-464a-b639-601f466dc787 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:51.757527 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf3177-ddf0-464a-b639-601f466dc787 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:51.757793 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf3177-ddf0-464a-b639-601f466dc787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:51.759439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-38bf3177-ddf0-464a-b639-601f466dc787 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:51.880070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf3177-ddf0-464a-b639-601f466dc787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:05:51.881465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf3177-ddf0-464a-b639-601f466dc787 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:05:51.901276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 670/1337] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:05:54.473655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adccb2-3a86-435f-a385-befdab469474 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:54.474200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adccb2-3a86-435f-a385-befdab469474 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:54.474200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adccb2-3a86-435f-a385-befdab469474 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:54.475479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 668/1338] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:05:54.481135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c7be3-61a9-4f86-8bfa-1b7c86f9cfdd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:54.481627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c7be3-61a9-4f86-8bfa-1b7c86f9cfdd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:54.481627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c7be3-61a9-4f86-8bfa-1b7c86f9cfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:54.553887 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5a5c7be3-61a9-4f86-8bfa-1b7c86f9cfdd None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:05:54.555112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5a5c7be3-61a9-4f86-8bfa-1b7c86f9cfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:54.564210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:54.657218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 671/1339] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:05:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:05:54.938607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adccb2-3a86-435f-a385-befdab469474 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:05:54.940198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d6adccb2-3a86-435f-a385-befdab469474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:55.051970 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adccb2-3a86-435f-a385-befdab469474 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:05:55.130655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6afdf6-a7ef-4933-b2ef-2ef7cb68ce15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:05:55.131264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6afdf6-a7ef-4933-b2ef-2ef7cb68ce15 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:55.131544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6afdf6-a7ef-4933-b2ef-2ef7cb68ce15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:55.131815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6afdf6-a7ef-4933-b2ef-2ef7cb68ce15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:55.133544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bc6afdf6-a7ef-4933-b2ef-2ef7cb68ce15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:55.173433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:55.251818 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6afdf6-a7ef-4933-b2ef-2ef7cb68ce15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:05:55.253316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6afdf6-a7ef-4933-b2ef-2ef7cb68ce15 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:05:55.272401 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 669/1340] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:05:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:05:59.469120 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962e370-25d7-43a0-b04d-18440d0a5a3f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:59.469845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962e370-25d7-43a0-b04d-18440d0a5a3f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:59.469845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962e370-25d7-43a0-b04d-18440d0a5a3f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:59.471010 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 672/1341] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:05:59.476818 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c454f-8bd6-4345-88c3-1dd697e69823 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:05:59.477337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c454f-8bd6-4345-88c3-1dd697e69823 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:05:59.477517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c454f-8bd6-4345-88c3-1dd697e69823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:05:59.551434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-918c454f-8bd6-4345-88c3-1dd697e69823 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:05:59.552688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-918c454f-8bd6-4345-88c3-1dd697e69823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:05:59.653215 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:05:59.664592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 670/1342] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:05:59.949022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962e370-25d7-43a0-b04d-18440d0a5a3f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:05:59.950675 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1962e370-25d7-43a0-b04d-18440d0a5a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:00.071583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962e370-25d7-43a0-b04d-18440d0a5a3f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:00.111492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:00.154813 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8259e9-fb18-4f05-9cd4-8339e9fcf395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:00.155469 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8259e9-fb18-4f05-9cd4-8339e9fcf395 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:00.155732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8259e9-fb18-4f05-9cd4-8339e9fcf395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:00.155999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8259e9-fb18-4f05-9cd4-8339e9fcf395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:00.157729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fe8259e9-fb18-4f05-9cd4-8339e9fcf395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:00.277776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8259e9-fb18-4f05-9cd4-8339e9fcf395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:00.279251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8259e9-fb18-4f05-9cd4-8339e9fcf395 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:00.299132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 673/1343] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:03.332448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c98ca9-7074-4cad-bdd2-862fcfcaf4f9 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:03.332915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c98ca9-7074-4cad-bdd2-862fcfcaf4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:03.333090 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c98ca9-7074-4cad-bdd2-862fcfcaf4f9 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:03.334570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 671/1344] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:03.340501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118308c-7942-4123-bcdb-c04e46d8f0b1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:03.341052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118308c-7942-4123-bcdb-c04e46d8f0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:03.341207 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118308c-7942-4123-bcdb-c04e46d8f0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:03.421134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8118308c-7942-4123-bcdb-c04e46d8f0b1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:03.422400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8118308c-7942-4123-bcdb-c04e46d8f0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:03.444525 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:03.526053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 674/1345] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:03.814383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c98ca9-7074-4cad-bdd2-862fcfcaf4f9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:03.815957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-17c98ca9-7074-4cad-bdd2-862fcfcaf4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:03.943599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c98ca9-7074-4cad-bdd2-862fcfcaf4f9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:04.019444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afda2d8-ce2b-4ea3-837f-b8469fe045a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:04.020056 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afda2d8-ce2b-4ea3-837f-b8469fe045a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:04.020336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afda2d8-ce2b-4ea3-837f-b8469fe045a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:04.020599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afda2d8-ce2b-4ea3-837f-b8469fe045a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:04.022232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0afda2d8-ce2b-4ea3-837f-b8469fe045a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:04.066243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:04.154086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afda2d8-ce2b-4ea3-837f-b8469fe045a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:04.155562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afda2d8-ce2b-4ea3-837f-b8469fe045a6 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:04.178345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 672/1346] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:08.612173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02345569-e5c4-4888-bd4e-3a689f0ab61f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:08.612769 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02345569-e5c4-4888-bd4e-3a689f0ab61f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:08.612769 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02345569-e5c4-4888-bd4e-3a689f0ab61f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:08.614112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 675/1347] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:08.619649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d0b76-7e73-46d1-9582-d3ecb282bf33 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:08.620035 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d0b76-7e73-46d1-9582-d3ecb282bf33 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:08.620188 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d0b76-7e73-46d1-9582-d3ecb282bf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:08.692061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1e3d0b76-7e73-46d1-9582-d3ecb282bf33 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:08.694226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1e3d0b76-7e73-46d1-9582-d3ecb282bf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:08.802071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:08.804702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 673/1348] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:09.096934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02345569-e5c4-4888-bd4e-3a689f0ab61f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:09.098484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-02345569-e5c4-4888-bd4e-3a689f0ab61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:09.221078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02345569-e5c4-4888-bd4e-3a689f0ab61f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:09.273107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:09.301463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c396e0-7fab-4a81-a921-f744768bf395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:09.302082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c396e0-7fab-4a81-a921-f744768bf395 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:09.302361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c396e0-7fab-4a81-a921-f744768bf395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:09.302632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c396e0-7fab-4a81-a921-f744768bf395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:09.304263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-09c396e0-7fab-4a81-a921-f744768bf395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:09.419230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c396e0-7fab-4a81-a921-f744768bf395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:09.420704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c396e0-7fab-4a81-a921-f744768bf395 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:09.439984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 676/1349] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:14.469370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c25ffc-9240-41cc-9a42-dbde18be966f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:14.469811 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c25ffc-9240-41cc-9a42-dbde18be966f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:14.469951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c25ffc-9240-41cc-9a42-dbde18be966f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:14.472297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 674/1350] 10.222.0.79 () {54 vars in 921 bytes} [Tue Sep 6 21:06:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:14.478506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a7a4e-eaf5-44f8-91e1-302d80b37fe2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:14.478506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a7a4e-eaf5-44f8-91e1-302d80b37fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:14.478954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a7a4e-eaf5-44f8-91e1-302d80b37fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:14.581416 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-850a7a4e-eaf5-44f8-91e1-302d80b37fe2 None None] MFA Rules not processed for user `c40b5ff21d1940749bf479cc936fc0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:14.582804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-850a7a4e-eaf5-44f8-91e1-302d80b37fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:14.635587 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:14.745616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 677/1351] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Sep 6 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:06:18.941035 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b986a3c-74dd-4126-8721-92526ba8c024 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:18.941699 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b986a3c-74dd-4126-8721-92526ba8c024 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:18.941699 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b986a3c-74dd-4126-8721-92526ba8c024 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:18.942897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 675/1352] 10.222.0.79 () {54 vars in 921 bytes} [Tue Sep 6 21:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:18.948301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e180d99-d087-48e6-bdde-091b3ea6953e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:18.948871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e180d99-d087-48e6-bdde-091b3ea6953e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:18.948956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e180d99-d087-48e6-bdde-091b3ea6953e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:19.045404 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5e180d99-d087-48e6-bdde-091b3ea6953e None None] MFA Rules not processed for user `c40b5ff21d1940749bf479cc936fc0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:19.048569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5e180d99-d087-48e6-bdde-091b3ea6953e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:19.124163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:19.163295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 678/1353] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Sep 6 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:20.638835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5748bd-a6b8-49e4-9a41-8a005ecba7d4 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:20.639435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5748bd-a6b8-49e4-9a41-8a005ecba7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:20.639435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5748bd-a6b8-49e4-9a41-8a005ecba7d4 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:20.640757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 676/1354] 10.222.0.79 () {54 vars in 921 bytes} [Tue Sep 6 21:06:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:20.646099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c41f37-2e34-4070-a36e-eb23fdcb0a88 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:20.646519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c41f37-2e34-4070-a36e-eb23fdcb0a88 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:20.646640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c41f37-2e34-4070-a36e-eb23fdcb0a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:20.728184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-38c41f37-2e34-4070-a36e-eb23fdcb0a88 None None] MFA Rules not processed for user `c40b5ff21d1940749bf479cc936fc0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:20.729840 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-38c41f37-2e34-4070-a36e-eb23fdcb0a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:20.849432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 679/1355] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Sep 6 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:24.258493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47e85c-9db0-449f-9798-f346051ea6b3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:24.259251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47e85c-9db0-449f-9798-f346051ea6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:24.259251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47e85c-9db0-449f-9798-f346051ea6b3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:24.261358 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 677/1356] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:24.269750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbf183-8bbc-44ca-89e3-7fbed11f7ff5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:24.270823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbf183-8bbc-44ca-89e3-7fbed11f7ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:24.271345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbf183-8bbc-44ca-89e3-7fbed11f7ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:24.318650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:24.364172 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ccbbf183-8bbc-44ca-89e3-7fbed11f7ff5 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:24.365684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ccbbf183-8bbc-44ca-89e3-7fbed11f7ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:24.494218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 680/1357] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:24.798516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfc37d-a3db-4922-a039-792bac9123c0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:24.799470 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfc37d-a3db-4922-a039-792bac9123c0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:24.800076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfc37d-a3db-4922-a039-792bac9123c0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:24.801995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 678/1358] 10.222.0.79 () {54 vars in 962 bytes} [Tue Sep 6 21:06:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:24.810437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc0510-36c8-40a0-9a56-1576ebd2b796 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:24.810437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc0510-36c8-40a0-9a56-1576ebd2b796 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:24.810437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc0510-36c8-40a0-9a56-1576ebd2b796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:24.907277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-76fc0510-36c8-40a0-9a56-1576ebd2b796 None None] MFA Rules not processed for user `12d51454d20c46d7a5e6d963a2f987cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:24.908811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-76fc0510-36c8-40a0-9a56-1576ebd2b796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:24.966009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:25.045023 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 681/1359] 10.222.0.79 () {58 vars in 1059 bytes} [Tue Sep 6 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:25.053366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfc37d-a3db-4922-a039-792bac9123c0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:25.054998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fedfc37d-a3db-4922-a039-792bac9123c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:25.218471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfc37d-a3db-4922-a039-792bac9123c0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:25.299444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:25.343924 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34f17f-34b2-4a0f-8a34-1ed5a7d92e9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:25.344809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34f17f-34b2-4a0f-8a34-1ed5a7d92e9a service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:25.345194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34f17f-34b2-4a0f-8a34-1ed5a7d92e9a service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:25.345542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34f17f-34b2-4a0f-8a34-1ed5a7d92e9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:25.347310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ab34f17f-34b2-4a0f-8a34-1ed5a7d92e9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:25.448880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34f17f-34b2-4a0f-8a34-1ed5a7d92e9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:25.451382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34f17f-34b2-4a0f-8a34-1ed5a7d92e9a service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:25.474721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 679/1360] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:27.906277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cba98-7023-4112-b267-6e4ee5f29867 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:27.906909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cba98-7023-4112-b267-6e4ee5f29867 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:27.906909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cba98-7023-4112-b267-6e4ee5f29867 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:27.908062 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 682/1361] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:27.913743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b170f5-6188-4fff-bda5-ac76bb891404 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:27.914710 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b170f5-6188-4fff-bda5-ac76bb891404 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:27.914710 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b170f5-6188-4fff-bda5-ac76bb891404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:27.983962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d2b170f5-6188-4fff-bda5-ac76bb891404 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:27.985222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d2b170f5-6188-4fff-bda5-ac76bb891404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:28.024547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:28.095351 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 680/1362] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:28.382966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cba98-7023-4112-b267-6e4ee5f29867 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:28.385131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-390cba98-7023-4112-b267-6e4ee5f29867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:28.583366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cba98-7023-4112-b267-6e4ee5f29867 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:28.597528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:28.744038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528d284-61a7-443e-a173-5449acd7481f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:28.744703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528d284-61a7-443e-a173-5449acd7481f service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:28.745022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528d284-61a7-443e-a173-5449acd7481f service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:28.745291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528d284-61a7-443e-a173-5449acd7481f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:28.747144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8528d284-61a7-443e-a173-5449acd7481f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:28.847278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528d284-61a7-443e-a173-5449acd7481f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:28.848883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528d284-61a7-443e-a173-5449acd7481f service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:28.868308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 683/1363] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:31.373325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d7a53-683e-4ce9-b568-9f90d7c19e23 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:31.373890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d7a53-683e-4ce9-b568-9f90d7c19e23 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:31.373890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d7a53-683e-4ce9-b568-9f90d7c19e23 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:31.375392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 681/1364] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:31.380798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228a3bc-4844-4dcc-b95e-29d3d8af2e96 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:31.381216 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228a3bc-4844-4dcc-b95e-29d3d8af2e96 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:31.381485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228a3bc-4844-4dcc-b95e-29d3d8af2e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:31.398862 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:31.453743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1228a3bc-4844-4dcc-b95e-29d3d8af2e96 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:31.454950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1228a3bc-4844-4dcc-b95e-29d3d8af2e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:31.558907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 684/1365] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:31.845582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2058d0-5bce-4368-bf7c-41c143b5cf8e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:31.845889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2058d0-5bce-4368-bf7c-41c143b5cf8e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:31.846140 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2058d0-5bce-4368-bf7c-41c143b5cf8e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:31.847666 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 682/1366] 10.222.0.79 () {54 vars in 962 bytes} [Tue Sep 6 21:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:31.853506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26b973-6f6c-47ee-839c-6ff9236e5954 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:31.853793 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26b973-6f6c-47ee-839c-6ff9236e5954 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:31.854090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26b973-6f6c-47ee-839c-6ff9236e5954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:31.935102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ea26b973-6f6c-47ee-839c-6ff9236e5954 None None] MFA Rules not processed for user `12d51454d20c46d7a5e6d963a2f987cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:31.936303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ea26b973-6f6c-47ee-839c-6ff9236e5954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:31.973749 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:32.066108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 685/1367] 10.222.0.79 () {58 vars in 1059 bytes} [Tue Sep 6 21:06:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:32.074370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2058d0-5bce-4368-bf7c-41c143b5cf8e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:32.075904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9a2058d0-5bce-4368-bf7c-41c143b5cf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:32.229568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2058d0-5bce-4368-bf7c-41c143b5cf8e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:32.252479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:32.355820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eab76b-4e13-40eb-863d-a743e50af5b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:32.356446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eab76b-4e13-40eb-863d-a743e50af5b0 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:32.356749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eab76b-4e13-40eb-863d-a743e50af5b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:32.357022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eab76b-4e13-40eb-863d-a743e50af5b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:32.358642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f4eab76b-4e13-40eb-863d-a743e50af5b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:32.453783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eab76b-4e13-40eb-863d-a743e50af5b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:32.456242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eab76b-4e13-40eb-863d-a743e50af5b0 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:32.475401 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 683/1368] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:32.611531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba924c-517d-4bd1-845c-f3d45a8fa159 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:32.612100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba924c-517d-4bd1-845c-f3d45a8fa159 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:32.612309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba924c-517d-4bd1-845c-f3d45a8fa159 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:32.614222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 686/1369] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:32.620171 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d346da-f21a-41c6-bd63-adf48a6c7b70 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:32.620442 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d346da-f21a-41c6-bd63-adf48a6c7b70 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:32.620785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d346da-f21a-41c6-bd63-adf48a6c7b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:32.658544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:32.709683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e5d346da-f21a-41c6-bd63-adf48a6c7b70 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:32.712150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e5d346da-f21a-41c6-bd63-adf48a6c7b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:32.854964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 684/1370] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:33.190657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a4e0a-44eb-409b-8b0c-b54693017384 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:33.191058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a4e0a-44eb-409b-8b0c-b54693017384 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:33.191427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a4e0a-44eb-409b-8b0c-b54693017384 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:33.193085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 687/1371] 10.222.0.79 () {54 vars in 964 bytes} [Tue Sep 6 21:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:33.198554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed482a-953c-4271-95a4-3450664298ce None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:33.198809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed482a-953c-4271-95a4-3450664298ce None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:33.199106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed482a-953c-4271-95a4-3450664298ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:33.286253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-89ed482a-953c-4271-95a4-3450664298ce None None] MFA Rules not processed for user `f50ae3b9997d4dc39c3d67ca40bbc566`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:33.287596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-89ed482a-953c-4271-95a4-3450664298ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:33.339118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:33.373171 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 685/1372] 10.222.0.79 () {58 vars in 1061 bytes} [Tue Sep 6 21:06:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:33.381225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a4e0a-44eb-409b-8b0c-b54693017384 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:33.382940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9e4a4e0a-44eb-409b-8b0c-b54693017384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:33.499972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a4e0a-44eb-409b-8b0c-b54693017384 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:33.544929 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:33.599153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599c366-70c2-4d07-9b9b-3946c47c5285 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:33.599153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599c366-70c2-4d07-9b9b-3946c47c5285 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:33.599153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599c366-70c2-4d07-9b9b-3946c47c5285 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:33.599153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599c366-70c2-4d07-9b9b-3946c47c5285 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:33.600636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e599c366-70c2-4d07-9b9b-3946c47c5285 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:33.720025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599c366-70c2-4d07-9b9b-3946c47c5285 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:33.721537 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599c366-70c2-4d07-9b9b-3946c47c5285 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:33.745382 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 688/1373] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:34.745963 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064c4bf-8597-4fa8-9d15-610bd1e68535 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:34.746511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064c4bf-8597-4fa8-9d15-610bd1e68535 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:34.746511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064c4bf-8597-4fa8-9d15-610bd1e68535 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:34.747810 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 686/1374] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:34.753221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb46f4-6da1-47f9-8615-5dd1c64e8efa None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:34.753645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb46f4-6da1-47f9-8615-5dd1c64e8efa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:34.753769 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb46f4-6da1-47f9-8615-5dd1c64e8efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:34.819473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-abfb46f4-6da1-47f9-8615-5dd1c64e8efa None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:34.820712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-abfb46f4-6da1-47f9-8615-5dd1c64e8efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:34.842138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:34.925350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 689/1375] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:35.216767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064c4bf-8597-4fa8-9d15-610bd1e68535 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:35.218414 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9064c4bf-8597-4fa8-9d15-610bd1e68535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:35.384361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064c4bf-8597-4fa8-9d15-610bd1e68535 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:35.438860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:35.505624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5eb79-7b1c-4c80-b1fd-49d3d880dabd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:35.506243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5eb79-7b1c-4c80-b1fd-49d3d880dabd service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:35.506517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5eb79-7b1c-4c80-b1fd-49d3d880dabd service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:35.506798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5eb79-7b1c-4c80-b1fd-49d3d880dabd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:35.508528 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-72c5eb79-7b1c-4c80-b1fd-49d3d880dabd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:35.616404 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5eb79-7b1c-4c80-b1fd-49d3d880dabd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:35.618289 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5eb79-7b1c-4c80-b1fd-49d3d880dabd service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:35.640185 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 687/1376] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:37.863845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5281a-4c61-40ee-9743-203d269d8265 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:37.864383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5281a-4c61-40ee-9743-203d269d8265 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:37.864383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5281a-4c61-40ee-9743-203d269d8265 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:37.865656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 690/1377] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:37.871839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e61b3-fdc0-4433-b9a2-09f2788b4931 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:37.872335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e61b3-fdc0-4433-b9a2-09f2788b4931 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:37.872438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e61b3-fdc0-4433-b9a2-09f2788b4931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:37.944868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7f2e61b3-fdc0-4433-b9a2-09f2788b4931 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:37.946009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7f2e61b3-fdc0-4433-b9a2-09f2788b4931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:37.953625 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:38.056751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 688/1378] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:38.343483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5281a-4c61-40ee-9743-203d269d8265 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:38.345037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-abd5281a-4c61-40ee-9743-203d269d8265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:38.491292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5281a-4c61-40ee-9743-203d269d8265 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:38.494989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:38.604438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf1b49-5b19-4d30-9604-d909724cfd8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:38.605098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf1b49-5b19-4d30-9604-d909724cfd8e service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:38.605374 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf1b49-5b19-4d30-9604-d909724cfd8e service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:38.605643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf1b49-5b19-4d30-9604-d909724cfd8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:38.607237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8ebf1b49-5b19-4d30-9604-d909724cfd8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:38.698579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf1b49-5b19-4d30-9604-d909724cfd8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:38.700195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf1b49-5b19-4d30-9604-d909724cfd8e service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:38.718559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 691/1379] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:40.923279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7ae82-1fb2-4b5e-be84-af9642691d3d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:40.923822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7ae82-1fb2-4b5e-be84-af9642691d3d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:40.923822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7ae82-1fb2-4b5e-be84-af9642691d3d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:40.925488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 689/1380] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:40.931734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974b56f-1770-490d-be44-fa9bb55f7082 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:40.932274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974b56f-1770-490d-be44-fa9bb55f7082 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:40.932355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974b56f-1770-490d-be44-fa9bb55f7082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:40.945776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:41.006260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0974b56f-1770-490d-be44-fa9bb55f7082 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:41.007711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0974b56f-1770-490d-be44-fa9bb55f7082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:41.156386 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 692/1381] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:41.453366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7ae82-1fb2-4b5e-be84-af9642691d3d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:41.455002 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-48b7ae82-1fb2-4b5e-be84-af9642691d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:41.625979 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:41.638240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7ae82-1fb2-4b5e-be84-af9642691d3d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:41.769492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e094d3-681a-4012-b8bc-91134af9df79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:41.770302 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e094d3-681a-4012-b8bc-91134af9df79 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:41.770665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e094d3-681a-4012-b8bc-91134af9df79 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:41.771092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e094d3-681a-4012-b8bc-91134af9df79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:41.773449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-19e094d3-681a-4012-b8bc-91134af9df79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:41.880257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e094d3-681a-4012-b8bc-91134af9df79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:41.882003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e094d3-681a-4012-b8bc-91134af9df79 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:41.893881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:41.903932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 690/1382] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:44.173271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f2d58-528c-415d-aa1b-4b9209ede8fb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:44.173827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f2d58-528c-415d-aa1b-4b9209ede8fb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:44.173827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f2d58-528c-415d-aa1b-4b9209ede8fb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:44.175167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 693/1383] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:44.181007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6c1cd-32c1-41c2-8504-d15e76b0e070 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:44.181443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6c1cd-32c1-41c2-8504-d15e76b0e070 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:44.181591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6c1cd-32c1-41c2-8504-d15e76b0e070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:44.254948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-80d6c1cd-32c1-41c2-8504-d15e76b0e070 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:44.256197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-80d6c1cd-32c1-41c2-8504-d15e76b0e070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:44.363129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 691/1384] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:44.655236 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f2d58-528c-415d-aa1b-4b9209ede8fb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:44.656922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0b9f2d58-528c-415d-aa1b-4b9209ede8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:44.774246 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:44.855454 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f2d58-528c-415d-aa1b-4b9209ede8fb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:45.010309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42930b1f-ef95-4137-b923-5723ddd24f6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:45.011221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42930b1f-ef95-4137-b923-5723ddd24f6c service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:45.011590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42930b1f-ef95-4137-b923-5723ddd24f6c service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:45.012043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42930b1f-ef95-4137-b923-5723ddd24f6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:45.014274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-42930b1f-ef95-4137-b923-5723ddd24f6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:45.070484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:45.118540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42930b1f-ef95-4137-b923-5723ddd24f6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:45.120780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42930b1f-ef95-4137-b923-5723ddd24f6c service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:45.144811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 694/1385] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:47.587299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e2464-497b-477a-984c-95b6a0d30b38 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:47.587898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e2464-497b-477a-984c-95b6a0d30b38 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:47.587898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e2464-497b-477a-984c-95b6a0d30b38 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:47.589165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 692/1386] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:47.594794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7d739-2477-4b48-b824-5650a75ee693 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:47.595258 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7d739-2477-4b48-b824-5650a75ee693 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:47.595357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7d739-2477-4b48-b824-5650a75ee693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:47.665773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-57b7d739-2477-4b48-b824-5650a75ee693 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:47.667063 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-57b7d739-2477-4b48-b824-5650a75ee693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:47.768045 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 695/1387] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:48.051636 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e2464-497b-477a-984c-95b6a0d30b38 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:48.053206 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-798e2464-497b-477a-984c-95b6a0d30b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:48.132000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:48.205921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e2464-497b-477a-984c-95b6a0d30b38 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:48.324937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7c31e-d4e1-4262-99e5-b7ed694ccd94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:48.325472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7c31e-d4e1-4262-99e5-b7ed694ccd94 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:48.325753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7c31e-d4e1-4262-99e5-b7ed694ccd94 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:48.326026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7c31e-d4e1-4262-99e5-b7ed694ccd94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:48.327661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-27f7c31e-d4e1-4262-99e5-b7ed694ccd94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:48.367626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:48.422639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7c31e-d4e1-4262-99e5-b7ed694ccd94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:48.424335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7c31e-d4e1-4262-99e5-b7ed694ccd94 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:48.444219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 693/1388] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:48.934068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c676487-fcc8-477d-92ce-00cb19dba81a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:48.934627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c676487-fcc8-477d-92ce-00cb19dba81a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:48.934627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c676487-fcc8-477d-92ce-00cb19dba81a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:48.936023 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 696/1389] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:48.941562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30d686-ed0f-4f4f-add2-34ba47dd5e13 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:48.942028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30d686-ed0f-4f4f-add2-34ba47dd5e13 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:48.942126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30d686-ed0f-4f4f-add2-34ba47dd5e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:49.014250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0f30d686-ed0f-4f4f-add2-34ba47dd5e13 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:49.015492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0f30d686-ed0f-4f4f-add2-34ba47dd5e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:49.116776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 694/1390] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:49.434591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbcff1-7444-4a30-a19b-5d4ad35e5de3 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:49.434872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbcff1-7444-4a30-a19b-5d4ad35e5de3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:49.435427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbcff1-7444-4a30-a19b-5d4ad35e5de3 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:49.437071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 697/1391] 10.222.0.79 () {54 vars in 961 bytes} [Tue Sep 6 21:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:49.442278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59873d6d-3361-4eb3-909e-470c6ff74135 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:49.442563 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59873d6d-3361-4eb3-909e-470c6ff74135 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:49.442836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59873d6d-3361-4eb3-909e-470c6ff74135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:49.459162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:49.521419 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-59873d6d-3361-4eb3-909e-470c6ff74135 None None] MFA Rules not processed for user `a76937ab52e445c59b9eb9638fd4a753`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:49.522706 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-59873d6d-3361-4eb3-909e-470c6ff74135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:49.604570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 695/1392] 10.222.0.79 () {58 vars in 1058 bytes} [Tue Sep 6 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:06:49.614241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbcff1-7444-4a30-a19b-5d4ad35e5de3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:49.615846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-85fbcff1-7444-4a30-a19b-5d4ad35e5de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:49.648282 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:49.726365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbcff1-7444-4a30-a19b-5d4ad35e5de3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:49.804714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7453c-8213-457a-b2f1-792cbe0b7708 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a76937ab52e445c59b9eb9638fd4a753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:49.805297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7453c-8213-457a-b2f1-792cbe0b7708 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:49.805570 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7453c-8213-457a-b2f1-792cbe0b7708 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:49.805836 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7453c-8213-457a-b2f1-792cbe0b7708 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:49.807442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-14f7453c-8213-457a-b2f1-792cbe0b7708 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:49.884203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:49.916943 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7453c-8213-457a-b2f1-792cbe0b7708 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:49.918396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7453c-8213-457a-b2f1-792cbe0b7708 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:49.937221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 698/1393] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:50.661626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9effa14-207d-449f-bb0f-7739afdd2f2e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:50.662266 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9effa14-207d-449f-bb0f-7739afdd2f2e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:50.662374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9effa14-207d-449f-bb0f-7739afdd2f2e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:50.664095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 696/1394] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:50.670711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a9ae3-8738-44c5-83b4-9adac51407e3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:50.671011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a9ae3-8738-44c5-83b4-9adac51407e3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:50.671288 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a9ae3-8738-44c5-83b4-9adac51407e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:50.699412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9200a67-d6f1-40bf-8625-ce4e09dae266 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:50.699700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9200a67-d6f1-40bf-8625-ce4e09dae266 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:50.699980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9200a67-d6f1-40bf-8625-ce4e09dae266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:50.746988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5e9a9ae3-8738-44c5-83b4-9adac51407e3 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:50.747939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5e9a9ae3-8738-44c5-83b4-9adac51407e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:50.770092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c9200a67-d6f1-40bf-8625-ce4e09dae266 None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:50.771312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c9200a67-d6f1-40bf-8625-ce4e09dae266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:50.830407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:50.852894 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 699/1395] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:50.865951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 697/1396] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:50.909305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c781f-fd43-431e-bfe8-f16054e6d3b4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:50.909660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c781f-fd43-431e-bfe8-f16054e6d3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:50.909935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c781f-fd43-431e-bfe8-f16054e6d3b4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:50.911284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 700/1397] 10.222.0.79 () {54 vars in 943 bytes} [Tue Sep 6 21:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:50.917821 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c06a9-a0b7-48cd-9897-89efff392efe None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:50.918108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c06a9-a0b7-48cd-9897-89efff392efe None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:50.918355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c06a9-a0b7-48cd-9897-89efff392efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:50.997171 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4a4c06a9-a0b7-48cd-9897-89efff392efe None None] MFA Rules not processed for user `061c584de8264868af8782402a98a72e`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:50.998406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4a4c06a9-a0b7-48cd-9897-89efff392efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:51.106008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 698/1398] 10.222.0.79 () {58 vars in 1040 bytes} [Tue Sep 6 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:51.115646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c781f-fd43-431e-bfe8-f16054e6d3b4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:51.117405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5b9c781f-fd43-431e-bfe8-f16054e6d3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:51.137967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c06a9-a0b7-48cd-9897-89efff392efe None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:51.139257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4a4c06a9-a0b7-48cd-9897-89efff392efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:51.159343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:51.183903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:51.252770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c06a9-a0b7-48cd-9897-89efff392efe None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:51.271677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c781f-fd43-431e-bfe8-f16054e6d3b4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:51.372044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd24e9-e465-4135-af19-d2a5f8a7115b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:51.372720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd24e9-e465-4135-af19-d2a5f8a7115b service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:51.373003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd24e9-e465-4135-af19-d2a5f8a7115b service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:51.373275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd24e9-e465-4135-af19-d2a5f8a7115b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:51.375125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-90bd24e9-e465-4135-af19-d2a5f8a7115b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:51.376596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11de82-36e9-47c9-9204-428138990896 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:51.377234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11de82-36e9-47c9-9204-428138990896 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:51.377503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11de82-36e9-47c9-9204-428138990896 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:51.377737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11de82-36e9-47c9-9204-428138990896 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:51.379347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ba11de82-36e9-47c9-9204-428138990896 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:51.401637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:51.418829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:51.464150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11de82-36e9-47c9-9204-428138990896 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:51.465823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11de82-36e9-47c9-9204-428138990896 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:51.470725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd24e9-e465-4135-af19-d2a5f8a7115b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:51.472342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd24e9-e465-4135-af19-d2a5f8a7115b service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:51.473696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 701/1399] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:06:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:06:51.491502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 699/1400] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:06:53.727598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da4c27-8df5-41ec-ad04-5b6438b4cbed None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:53.728185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da4c27-8df5-41ec-ad04-5b6438b4cbed None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:53.728185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da4c27-8df5-41ec-ad04-5b6438b4cbed None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:53.733884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 702/1401] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:53.735544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56536fee-65b4-4867-9974-152732b91fad None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:53.736035 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56536fee-65b4-4867-9974-152732b91fad None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:53.736127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56536fee-65b4-4867-9974-152732b91fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:53.815582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-56536fee-65b4-4867-9974-152732b91fad None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:53.816857 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-56536fee-65b4-4867-9974-152732b91fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:53.927168 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 700/1402] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:06:54.220061 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da4c27-8df5-41ec-ad04-5b6438b4cbed None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:54.221654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e7da4c27-8df5-41ec-ad04-5b6438b4cbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:54.387164 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da4c27-8df5-41ec-ad04-5b6438b4cbed None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:54.411431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:54.519751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610f3b8-0bb2-4cf8-9400-907e744b4f9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:54.520372 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610f3b8-0bb2-4cf8-9400-907e744b4f9d service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:54.520649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610f3b8-0bb2-4cf8-9400-907e744b4f9d service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:54.520942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610f3b8-0bb2-4cf8-9400-907e744b4f9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:54.522565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9610f3b8-0bb2-4cf8-9400-907e744b4f9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:54.626947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9610f3b8-0bb2-4cf8-9400-907e744b4f9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:54.628720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9610f3b8-0bb2-4cf8-9400-907e744b4f9d service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:54.654722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 703/1403] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:06:56.888880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c44d1-6bb8-4a94-a199-e3beaeefc5e9 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:56.889415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c44d1-6bb8-4a94-a199-e3beaeefc5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:56.889415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c44d1-6bb8-4a94-a199-e3beaeefc5e9 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:56.891147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 701/1404] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:06:56.896212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6232a5-4979-41b4-90a9-31d2ca78ee03 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:56.896687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6232a5-4979-41b4-90a9-31d2ca78ee03 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:56.896780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6232a5-4979-41b4-90a9-31d2ca78ee03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:56.924694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:56.973635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4a6232a5-4979-41b4-90a9-31d2ca78ee03 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:06:56.975209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4a6232a5-4979-41b4-90a9-31d2ca78ee03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:57.096018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 704/1405] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:06:57.385712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c44d1-6bb8-4a94-a199-e3beaeefc5e9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:06:57.387256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-041c44d1-6bb8-4a94-a199-e3beaeefc5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:57.412635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:57.555332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c44d1-6bb8-4a94-a199-e3beaeefc5e9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:06:57.659257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:06:57.685904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1930ac7-7c40-42de-9c55-bbc5884533e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:06:57.686520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1930ac7-7c40-42de-9c55-bbc5884533e4 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:06:57.686802 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1930ac7-7c40-42de-9c55-bbc5884533e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:06:57.687079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1930ac7-7c40-42de-9c55-bbc5884533e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:06:57.688755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b1930ac7-7c40-42de-9c55-bbc5884533e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:06:57.788590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1930ac7-7c40-42de-9c55-bbc5884533e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:06:57.790703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1930ac7-7c40-42de-9c55-bbc5884533e4 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:06:57.812919 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 702/1406] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:00.050906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cb9d1-2958-4987-9c76-2f036731b1e5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:00.051509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cb9d1-2958-4987-9c76-2f036731b1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:00.051618 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cb9d1-2958-4987-9c76-2f036731b1e5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:00.053652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 705/1407] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:07:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:00.059497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42037b45-ae3a-44c4-a56c-f09f5cd274f3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:00.059965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42037b45-ae3a-44c4-a56c-f09f5cd274f3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:00.060081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42037b45-ae3a-44c4-a56c-f09f5cd274f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:00.130930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-42037b45-ae3a-44c4-a56c-f09f5cd274f3 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:00.132150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-42037b45-ae3a-44c4-a56c-f09f5cd274f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:00.192074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:00.248749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 703/1408] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:00.535930 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4cb9d1-2958-4987-9c76-2f036731b1e5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:00.537502 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4d4cb9d1-2958-4987-9c76-2f036731b1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:00.652952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:00.712619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4cb9d1-2958-4987-9c76-2f036731b1e5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:00.835588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a4d8d-0514-4c64-9721-2abc4f98c1d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:00.836108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a4d8d-0514-4c64-9721-2abc4f98c1d1 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:00.836392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a4d8d-0514-4c64-9721-2abc4f98c1d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:00.836692 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a4d8d-0514-4c64-9721-2abc4f98c1d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:00.838477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e91a4d8d-0514-4c64-9721-2abc4f98c1d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:00.907346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:00.941677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a4d8d-0514-4c64-9721-2abc4f98c1d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:00.943340 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a4d8d-0514-4c64-9721-2abc4f98c1d1 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:00.963506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 706/1409] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:10.247114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=19 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:10.959772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=18 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:14.470177 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3bc83-ce8d-4d24-81e9-1ac408a0e2c6 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:14.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3bc83-ce8d-4d24-81e9-1ac408a0e2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:14.470785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3bc83-ce8d-4d24-81e9-1ac408a0e2c6 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:14.472104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 704/1410] 10.222.0.79 () {54 vars in 918 bytes} [Tue Sep 6 21:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:14.479969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6f578-60e6-47a2-8918-8a63adfa8bb4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:14.480407 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6f578-60e6-47a2-8918-8a63adfa8bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:14.480600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6f578-60e6-47a2-8918-8a63adfa8bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:14.575292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f3e6f578-60e6-47a2-8918-8a63adfa8bb4 None None] MFA Rules not processed for user `c40b5ff21d1940749bf479cc936fc0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:14.576576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f3e6f578-60e6-47a2-8918-8a63adfa8bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:14.688927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 707/1411] 10.222.0.79 () {58 vars in 1015 bytes} [Tue Sep 6 21:07:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:14.704038 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290cf39-e0d3-4ab2-a415-08cb7e51243f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:14.704322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290cf39-e0d3-4ab2-a415-08cb7e51243f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:14.704589 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290cf39-e0d3-4ab2-a415-08cb7e51243f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:14.705942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 705/1412] 10.222.0.79 () {54 vars in 918 bytes} [Tue Sep 6 21:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:14.711251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa74c02-c0b0-43b8-8823-baff8f65affa None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:14.711537 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa74c02-c0b0-43b8-8823-baff8f65affa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:14.711826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa74c02-c0b0-43b8-8823-baff8f65affa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:14.788206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:14.791165 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0fa74c02-c0b0-43b8-8823-baff8f65affa None None] MFA Rules not processed for user `12d51454d20c46d7a5e6d963a2f987cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:14.793517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0fa74c02-c0b0-43b8-8823-baff8f65affa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:14.918008 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 708/1413] 10.222.0.79 () {58 vars in 1015 bytes} [Tue Sep 6 21:07:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:17.520090 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f2c3-27ca-4dd9-93ab-3c437caa7708 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:17.520683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f2c3-27ca-4dd9-93ab-3c437caa7708 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:17.520683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f2c3-27ca-4dd9-93ab-3c437caa7708 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:17.522024 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:17.528699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad23c4-0c5e-4ba2-8c0b-601d85ca123e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:17.529856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad23c4-0c5e-4ba2-8c0b-601d85ca123e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:17.530319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad23c4-0c5e-4ba2-8c0b-601d85ca123e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:17.604040 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-00ad23c4-0c5e-4ba2-8c0b-601d85ca123e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:17.605310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-00ad23c4-0c5e-4ba2-8c0b-601d85ca123e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:17.664679 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:17.709405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:18.007306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8f2c3-27ca-4dd9-93ab-3c437caa7708 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:18.008890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-efd8f2c3-27ca-4dd9-93ab-3c437caa7708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:18.195073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8f2c3-27ca-4dd9-93ab-3c437caa7708 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:18.284213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:18.333580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59395f-d9ed-48b8-bfc6-2cc32cc8899a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:18.334194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59395f-d9ed-48b8-bfc6-2cc32cc8899a service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:18.334469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59395f-d9ed-48b8-bfc6-2cc32cc8899a service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:18.334737 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59395f-d9ed-48b8-bfc6-2cc32cc8899a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:18.336351 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bf59395f-d9ed-48b8-bfc6-2cc32cc8899a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:18.434585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59395f-d9ed-48b8-bfc6-2cc32cc8899a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:18.436254 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59395f-d9ed-48b8-bfc6-2cc32cc8899a service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:18.460859 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 707/1416] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:19.631366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c870025-1e5d-497f-9e00-842d07c0473f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:19.631901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c870025-1e5d-497f-9e00-842d07c0473f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:19.631901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c870025-1e5d-497f-9e00-842d07c0473f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:19.633841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 21:07:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:19.640628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b8c3f-82c7-4f2b-9729-b4b13e6c7408 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:19.641159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b8c3f-82c7-4f2b-9729-b4b13e6c7408 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:19.641159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b8c3f-82c7-4f2b-9729-b4b13e6c7408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:19.727582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-320b8c3f-82c7-4f2b-9729-b4b13e6c7408 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:19.728835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-320b8c3f-82c7-4f2b-9729-b4b13e6c7408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:19.758195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:19.829242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 21:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:19.987218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c870025-1e5d-497f-9e00-842d07c0473f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:19.988786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6c870025-1e5d-497f-9e00-842d07c0473f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:20.156420 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c870025-1e5d-497f-9e00-842d07c0473f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:20.208530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:20.298393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2d6c8-2459-4891-9b4e-d7e7bb377f2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:20.299006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2d6c8-2459-4891-9b4e-d7e7bb377f2e service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:20.299278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2d6c8-2459-4891-9b4e-d7e7bb377f2e service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:20.299544 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2d6c8-2459-4891-9b4e-d7e7bb377f2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:20.301199 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-12f2d6c8-2459-4891-9b4e-d7e7bb377f2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:20.400591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2d6c8-2459-4891-9b4e-d7e7bb377f2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:20.402261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2d6c8-2459-4891-9b4e-d7e7bb377f2e service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:20.425265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 711/1419] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:21.572466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a46fa9-f253-4e2f-85fb-641b05f0ed76 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:21.573071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a46fa9-f253-4e2f-85fb-641b05f0ed76 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:21.573071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a46fa9-f253-4e2f-85fb-641b05f0ed76 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:21.574308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 21:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:21.581711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7615a9-d753-438e-8629-a02571667cc1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:21.582456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7615a9-d753-438e-8629-a02571667cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:21.582716 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7615a9-d753-438e-8629-a02571667cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:21.646135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:21.670943 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4a7615a9-d753-438e-8629-a02571667cc1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:21.672204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4a7615a9-d753-438e-8629-a02571667cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:21.779358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 21:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:21.936529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a46fa9-f253-4e2f-85fb-641b05f0ed76 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:21.938001 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-60a46fa9-f253-4e2f-85fb-641b05f0ed76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:22.097464 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a46fa9-f253-4e2f-85fb-641b05f0ed76 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:22.110427 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:22.226702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b2d98-4645-42f9-88df-2a81e8c2988c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:22.227461 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b2d98-4645-42f9-88df-2a81e8c2988c service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:22.227849 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b2d98-4645-42f9-88df-2a81e8c2988c service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:22.228272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b2d98-4645-42f9-88df-2a81e8c2988c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:22.230865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c17b2d98-4645-42f9-88df-2a81e8c2988c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:22.335039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b2d98-4645-42f9-88df-2a81e8c2988c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:22.337312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b2d98-4645-42f9-88df-2a81e8c2988c service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:22.356697 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 710/1422] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:23.481426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b82f6c-9fa7-43e3-8fde-bd0bb336fbb6 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:23.482057 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b82f6c-9fa7-43e3-8fde-bd0bb336fbb6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:23.482057 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b82f6c-9fa7-43e3-8fde-bd0bb336fbb6 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:23.483303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 21:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:23.491808 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73557350-a993-4d54-88ca-07187ec6afb6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:23.492595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73557350-a993-4d54-88ca-07187ec6afb6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:23.492827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73557350-a993-4d54-88ca-07187ec6afb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:23.519207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:23.703370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-73557350-a993-4d54-88ca-07187ec6afb6 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:23.704616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-73557350-a993-4d54-88ca-07187ec6afb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:23.806616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 21:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:23.974593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b82f6c-9fa7-43e3-8fde-bd0bb336fbb6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:23.976721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f9b82f6c-9fa7-43e3-8fde-bd0bb336fbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:24.123606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:24.161588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b82f6c-9fa7-43e3-8fde-bd0bb336fbb6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:24.279244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33d6d1-060d-4fe7-a0bb-3afad92fdb96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:24.279853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33d6d1-060d-4fe7-a0bb-3afad92fdb96 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:24.280125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33d6d1-060d-4fe7-a0bb-3afad92fdb96 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:24.280391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33d6d1-060d-4fe7-a0bb-3afad92fdb96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:24.281998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bc33d6d1-060d-4fe7-a0bb-3afad92fdb96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:24.371088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:24.389867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33d6d1-060d-4fe7-a0bb-3afad92fdb96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:24.391504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33d6d1-060d-4fe7-a0bb-3afad92fdb96 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:24.410292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 714/1425] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:25.540875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5c87b-e2df-4b23-936e-19683b9d7a6c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:25.541460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5c87b-e2df-4b23-936e-19683b9d7a6c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:25.541460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5c87b-e2df-4b23-936e-19683b9d7a6c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:25.542854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 21:07:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:25.550390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0aefd4-91f2-49a5-85e2-c661ff00633a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:25.550829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0aefd4-91f2-49a5-85e2-c661ff00633a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:25.551163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0aefd4-91f2-49a5-85e2-c661ff00633a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:25.643772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fe0aefd4-91f2-49a5-85e2-c661ff00633a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:25.645034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fe0aefd4-91f2-49a5-85e2-c661ff00633a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:25.755575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 21:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:25.908781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5c87b-e2df-4b23-936e-19683b9d7a6c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:25.910234 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-53a5c87b-e2df-4b23-936e-19683b9d7a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:26.008256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:26.080469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5c87b-e2df-4b23-936e-19683b9d7a6c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:26.217078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3984ed-fe43-44f0-bb48-3bb059087d05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:26.217610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3984ed-fe43-44f0-bb48-3bb059087d05 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:26.217913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3984ed-fe43-44f0-bb48-3bb059087d05 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:26.218160 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3984ed-fe43-44f0-bb48-3bb059087d05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:26.219925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cc3984ed-fe43-44f0-bb48-3bb059087d05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:26.270639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:26.328840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3984ed-fe43-44f0-bb48-3bb059087d05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:26.330460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3984ed-fe43-44f0-bb48-3bb059087d05 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:26.349997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 713/1428] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:27.414197 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66e391-9e7d-4889-8ec3-a889d841761f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:27.414897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66e391-9e7d-4889-8ec3-a889d841761f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:27.414897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66e391-9e7d-4889-8ec3-a889d841761f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:27.416171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 21:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:27.423929 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a7f9-1131-44ca-a83f-27c23860d016 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:27.424743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a7f9-1131-44ca-a83f-27c23860d016 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:27.424928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a7f9-1131-44ca-a83f-27c23860d016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:27.506688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f5b5a7f9-1131-44ca-a83f-27c23860d016 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:27.507906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f5b5a7f9-1131-44ca-a83f-27c23860d016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:27.615074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 21:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:27.771109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66e391-9e7d-4889-8ec3-a889d841761f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:27.772645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8d66e391-9e7d-4889-8ec3-a889d841761f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:27.827253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:27.942105 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66e391-9e7d-4889-8ec3-a889d841761f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:28.069151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a014e-8d1f-4e76-9559-5badcf6a2fd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:28.069748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a014e-8d1f-4e76-9559-5badcf6a2fd2 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:28.070016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a014e-8d1f-4e76-9559-5badcf6a2fd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:28.070306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a014e-8d1f-4e76-9559-5badcf6a2fd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:28.071999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cc9a014e-8d1f-4e76-9559-5badcf6a2fd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:28.079178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:28.185608 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a014e-8d1f-4e76-9559-5badcf6a2fd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:28.187290 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a014e-8d1f-4e76-9559-5badcf6a2fd2 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:28.209348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 717/1431] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:29.272477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228a955-f030-4876-b7d4-e5ffdd8069ff None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:29.273136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228a955-f030-4876-b7d4-e5ffdd8069ff None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:29.273136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228a955-f030-4876-b7d4-e5ffdd8069ff None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:29.274559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 21:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:29.282449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ed681-f7a6-4bd4-b774-8e72778685d0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:29.283244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ed681-f7a6-4bd4-b774-8e72778685d0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:29.283244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ed681-f7a6-4bd4-b774-8e72778685d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:29.370093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-249ed681-f7a6-4bd4-b774-8e72778685d0 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:29.371310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-249ed681-f7a6-4bd4-b774-8e72778685d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:29.444950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:29.475899 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 21:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:29.630499 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228a955-f030-4876-b7d4-e5ffdd8069ff None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:29.632008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2228a955-f030-4876-b7d4-e5ffdd8069ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:29.651894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:29.806835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228a955-f030-4876-b7d4-e5ffdd8069ff None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:29.908896 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:29.935277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2119f6f-0d2c-41bf-9fc8-1b8af58547fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:29.935887 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119f6f-0d2c-41bf-9fc8-1b8af58547fb service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:29.936158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119f6f-0d2c-41bf-9fc8-1b8af58547fb service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:29.936426 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119f6f-0d2c-41bf-9fc8-1b8af58547fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:29.938051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a2119f6f-0d2c-41bf-9fc8-1b8af58547fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:30.029007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2119f6f-0d2c-41bf-9fc8-1b8af58547fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:30.030602 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2119f6f-0d2c-41bf-9fc8-1b8af58547fb service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:30.051362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 716/1434] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:31.125622 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7ee19-159e-4b55-9480-9221df7d134a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:31.126230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7ee19-159e-4b55-9480-9221df7d134a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:31.126230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7ee19-159e-4b55-9480-9221df7d134a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:31.127542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 21:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:31.135398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd9ccb-29f4-4caa-b0f4-91e9855f3050 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:31.136264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd9ccb-29f4-4caa-b0f4-91e9855f3050 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:31.136264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd9ccb-29f4-4caa-b0f4-91e9855f3050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:31.225654 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f1bd9ccb-29f4-4caa-b0f4-91e9855f3050 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:31.227413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f1bd9ccb-29f4-4caa-b0f4-91e9855f3050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:31.277448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:31.332552 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 21:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:31.482300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7ee19-159e-4b55-9480-9221df7d134a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:31.483738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-90f7ee19-159e-4b55-9480-9221df7d134a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:31.654256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7ee19-159e-4b55-9480-9221df7d134a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:31.713884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:31.773989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbfac5-0cc6-44c2-8246-32c5cfe7fd8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:31.774600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbfac5-0cc6-44c2-8246-32c5cfe7fd8d service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:31.774908 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbfac5-0cc6-44c2-8246-32c5cfe7fd8d service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:31.775179 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbfac5-0cc6-44c2-8246-32c5cfe7fd8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:31.776829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-39cbfac5-0cc6-44c2-8246-32c5cfe7fd8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:31.876347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbfac5-0cc6-44c2-8246-32c5cfe7fd8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:31.877989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbfac5-0cc6-44c2-8246-32c5cfe7fd8d service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:31.897332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 720/1437] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:32.958645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad145d56-1af5-4016-819d-17382636e0c5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:32.959552 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad145d56-1af5-4016-819d-17382636e0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:32.959552 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad145d56-1af5-4016-819d-17382636e0c5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:32.961753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 21:07:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:32.969093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e47cf-a19f-4457-8def-7ea75ad3f478 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:32.969587 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e47cf-a19f-4457-8def-7ea75ad3f478 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:32.969696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e47cf-a19f-4457-8def-7ea75ad3f478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:33.066576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0f1e47cf-a19f-4457-8def-7ea75ad3f478 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:33.067817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0f1e47cf-a19f-4457-8def-7ea75ad3f478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:33.069881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:33.175540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 21:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:33.327271 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad145d56-1af5-4016-819d-17382636e0c5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:33.329677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ad145d56-1af5-4016-819d-17382636e0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:33.499663 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad145d56-1af5-4016-819d-17382636e0c5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:33.527376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:33.619925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79574b98-53fa-4a56-930d-65af23f62d4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:33.620525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574b98-53fa-4a56-930d-65af23f62d4d service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:33.620829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574b98-53fa-4a56-930d-65af23f62d4d service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:33.621095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574b98-53fa-4a56-930d-65af23f62d4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:33.622681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-79574b98-53fa-4a56-930d-65af23f62d4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:33.720822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79574b98-53fa-4a56-930d-65af23f62d4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:33.722421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79574b98-53fa-4a56-930d-65af23f62d4d service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:33.760224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 719/1440] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:35.829346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ad2d-2e21-489d-8241-3653a95e42ba None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:35.829918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ad2d-2e21-489d-8241-3653a95e42ba None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:35.829918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ad2d-2e21-489d-8241-3653a95e42ba None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:35.831221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:35.836846 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e659632-f4c7-412d-ab88-b14806294dc9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:35.837454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e659632-f4c7-412d-ab88-b14806294dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:35.837454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e659632-f4c7-412d-ab88-b14806294dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:35.882695 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:35.911642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9e659632-f4c7-412d-ab88-b14806294dc9 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:35.912934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9e659632-f4c7-412d-ab88-b14806294dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:36.016703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:36.304208 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6ad2d-2e21-489d-8241-3653a95e42ba None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:36.307087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e9a6ad2d-2e21-489d-8241-3653a95e42ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:36.416333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6ad2d-2e21-489d-8241-3653a95e42ba None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:36.449623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:36.499627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa54dd1-8a91-4921-b153-c0a6ee3cdc58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:36.500262 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54dd1-8a91-4921-b153-c0a6ee3cdc58 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:36.500558 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54dd1-8a91-4921-b153-c0a6ee3cdc58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:36.503419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54dd1-8a91-4921-b153-c0a6ee3cdc58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:36.503419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ffa54dd1-8a91-4921-b153-c0a6ee3cdc58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:36.628158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa54dd1-8a91-4921-b153-c0a6ee3cdc58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:36.629645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa54dd1-8a91-4921-b153-c0a6ee3cdc58 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:36.649459 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 723/1443] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:39.041704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06a353-5ddc-42e7-a8ce-5688d314ce40 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:39.042291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06a353-5ddc-42e7-a8ce-5688d314ce40 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:39.042291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06a353-5ddc-42e7-a8ce-5688d314ce40 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:39.044322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:07:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:39.053223 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8751d-0c1b-45f5-9ae4-498f6183bd72 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:39.053642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8751d-0c1b-45f5-9ae4-498f6183bd72 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:39.054041 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8751d-0c1b-45f5-9ae4-498f6183bd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:39.137058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-eed8751d-0c1b-45f5-9ae4-498f6183bd72 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:39.138278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-eed8751d-0c1b-45f5-9ae4-498f6183bd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:39.152185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:39.244467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:39.534977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06a353-5ddc-42e7-a8ce-5688d314ce40 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:39.538712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6f06a353-5ddc-42e7-a8ce-5688d314ce40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:39.664460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06a353-5ddc-42e7-a8ce-5688d314ce40 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:39.676873 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:39.746456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864297a4-9174-44b5-99f7-4dc7b9b13d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:39.747186 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864297a4-9174-44b5-99f7-4dc7b9b13d51 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:39.747567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864297a4-9174-44b5-99f7-4dc7b9b13d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:39.747963 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864297a4-9174-44b5-99f7-4dc7b9b13d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:39.750676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-864297a4-9174-44b5-99f7-4dc7b9b13d51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:39.867884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864297a4-9174-44b5-99f7-4dc7b9b13d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:39.869330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864297a4-9174-44b5-99f7-4dc7b9b13d51 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:39.889387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 722/1446] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:42.396181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbce20-9502-43dc-b643-0ac181218bcb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:42.397096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbce20-9502-43dc-b643-0ac181218bcb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:42.397096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbce20-9502-43dc-b643-0ac181218bcb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:42.399208 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:07:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:42.407815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67da01e-6863-404b-83ff-2c3b78478c96 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:42.408521 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67da01e-6863-404b-83ff-2c3b78478c96 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:42.408694 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67da01e-6863-404b-83ff-2c3b78478c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:42.470678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:42.490308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f67da01e-6863-404b-83ff-2c3b78478c96 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:42.491537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f67da01e-6863-404b-83ff-2c3b78478c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:42.601571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:42.886871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbce20-9502-43dc-b643-0ac181218bcb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:42.888439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8bfbce20-9502-43dc-b643-0ac181218bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:42.999841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbce20-9502-43dc-b643-0ac181218bcb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:43.041157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:43.080965 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49649ef4-bfc8-4538-9af2-9f4ee9824441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:43.081574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49649ef4-bfc8-4538-9af2-9f4ee9824441 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:43.081853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49649ef4-bfc8-4538-9af2-9f4ee9824441 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:43.082119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49649ef4-bfc8-4538-9af2-9f4ee9824441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:43.083715 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-49649ef4-bfc8-4538-9af2-9f4ee9824441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:43.214206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49649ef4-bfc8-4538-9af2-9f4ee9824441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:43.215572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49649ef4-bfc8-4538-9af2-9f4ee9824441 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:43.236209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 726/1449] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:45.523673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae2ae5-26ca-444a-a3ab-08519f268f3d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:45.524591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae2ae5-26ca-444a-a3ab-08519f268f3d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:45.524713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae2ae5-26ca-444a-a3ab-08519f268f3d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:45.526126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:07:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:45.535097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869377b0-c1f9-4f00-bc6e-a91d2f302131 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:45.535788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869377b0-c1f9-4f00-bc6e-a91d2f302131 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:45.535938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869377b0-c1f9-4f00-bc6e-a91d2f302131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:45.617500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-869377b0-c1f9-4f00-bc6e-a91d2f302131 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:45.618879 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-869377b0-c1f9-4f00-bc6e-a91d2f302131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:45.641158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:45.728554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:46.015075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae2ae5-26ca-444a-a3ab-08519f268f3d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:46.016625 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a7ae2ae5-26ca-444a-a3ab-08519f268f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:46.126092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae2ae5-26ca-444a-a3ab-08519f268f3d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:46.146874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:46.213579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa398d4-9b23-488b-9af3-b3cf457d1d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:46.214676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa398d4-9b23-488b-9af3-b3cf457d1d06 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:46.214676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa398d4-9b23-488b-9af3-b3cf457d1d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:46.214676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa398d4-9b23-488b-9af3-b3cf457d1d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:46.216327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ffa398d4-9b23-488b-9af3-b3cf457d1d06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:46.340100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa398d4-9b23-488b-9af3-b3cf457d1d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:46.341602 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa398d4-9b23-488b-9af3-b3cf457d1d06 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:46.361170 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 725/1452] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:48.653856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055461d-3eaf-47b9-96a8-17f49fc74c5f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:48.654475 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055461d-3eaf-47b9-96a8-17f49fc74c5f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:48.654475 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055461d-3eaf-47b9-96a8-17f49fc74c5f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:48.655763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:48.662320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20076609-62e9-4718-abe7-94f3800908dd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:48.662753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20076609-62e9-4718-abe7-94f3800908dd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:48.662867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20076609-62e9-4718-abe7-94f3800908dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:48.736372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:48.740938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-20076609-62e9-4718-abe7-94f3800908dd None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:48.742393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-20076609-62e9-4718-abe7-94f3800908dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:48.848297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:49.131064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055461d-3eaf-47b9-96a8-17f49fc74c5f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:49.133777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b055461d-3eaf-47b9-96a8-17f49fc74c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:49.242014 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055461d-3eaf-47b9-96a8-17f49fc74c5f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:49.296401 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:49.323556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd7397-a205-4de3-9603-6a8c7f213dde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:49.324181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd7397-a205-4de3-9603-6a8c7f213dde service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:49.324459 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd7397-a205-4de3-9603-6a8c7f213dde service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:49.324762 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd7397-a205-4de3-9603-6a8c7f213dde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:49.326383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-37cd7397-a205-4de3-9603-6a8c7f213dde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:49.444265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd7397-a205-4de3-9603-6a8c7f213dde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:49.445672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd7397-a205-4de3-9603-6a8c7f213dde service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:49.465651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 729/1455] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:51.704642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67d5e4-845e-48e3-9e85-60b54b470cc0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:51.705236 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67d5e4-845e-48e3-9e85-60b54b470cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:51.705236 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67d5e4-845e-48e3-9e85-60b54b470cc0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:51.706571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:51.713183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736faf3-028e-42d8-964c-c7612becac09 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:51.713674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736faf3-028e-42d8-964c-c7612becac09 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:51.713789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736faf3-028e-42d8-964c-c7612becac09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:51.785804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9736faf3-028e-42d8-964c-c7612becac09 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:51.787027 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9736faf3-028e-42d8-964c-c7612becac09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:51.820135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:51.894048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:52.180314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67d5e4-845e-48e3-9e85-60b54b470cc0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:52.183222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6c67d5e4-845e-48e3-9e85-60b54b470cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:52.302633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67d5e4-845e-48e3-9e85-60b54b470cc0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:52.331395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:52.399993 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6583f7-767a-4a44-a4c6-e46638c66144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:52.400599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6583f7-767a-4a44-a4c6-e46638c66144 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:52.400902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6583f7-767a-4a44-a4c6-e46638c66144 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:52.401138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6583f7-767a-4a44-a4c6-e46638c66144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:52.402777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9b6583f7-767a-4a44-a4c6-e46638c66144 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:52.519714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6583f7-767a-4a44-a4c6-e46638c66144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:52.521199 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6583f7-767a-4a44-a4c6-e46638c66144 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:52.540322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 728/1458] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:54.830371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6242ef0-08cf-4f97-be73-01cc7dcfd36d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:54.831360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6242ef0-08cf-4f97-be73-01cc7dcfd36d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:54.831360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6242ef0-08cf-4f97-be73-01cc7dcfd36d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:54.833585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:07:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:54.842765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4620404-aed2-43a8-8c0c-bef7d08665f4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:54.843453 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4620404-aed2-43a8-8c0c-bef7d08665f4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:54.843453 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4620404-aed2-43a8-8c0c-bef7d08665f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:54.923867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e4620404-aed2-43a8-8c0c-bef7d08665f4 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:54.925135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e4620404-aed2-43a8-8c0c-bef7d08665f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:54.934190 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:55.028328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:55.310446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6242ef0-08cf-4f97-be73-01cc7dcfd36d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:55.311698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a6242ef0-08cf-4f97-be73-01cc7dcfd36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:55.434766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6242ef0-08cf-4f97-be73-01cc7dcfd36d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:55.496052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:55.515000 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12b3a7-76b8-4dae-96c2-0b4b69407561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:55.515609 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12b3a7-76b8-4dae-96c2-0b4b69407561 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:55.515889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12b3a7-76b8-4dae-96c2-0b4b69407561 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:55.516146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12b3a7-76b8-4dae-96c2-0b4b69407561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:55.517825 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ef12b3a7-76b8-4dae-96c2-0b4b69407561 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:55.632813 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12b3a7-76b8-4dae-96c2-0b4b69407561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:55.634388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12b3a7-76b8-4dae-96c2-0b4b69407561 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:55.653564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 732/1461] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:07:57.865288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37fa14-cf19-4f67-894a-5730e653e4a4 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:57.865850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37fa14-cf19-4f67-894a-5730e653e4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:57.865850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37fa14-cf19-4f67-894a-5730e653e4a4 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:57.867304 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:07:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:07:57.875331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cb496-16b7-4b27-8c50-f5d94cc62b40 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:57.875780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cb496-16b7-4b27-8c50-f5d94cc62b40 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:57.875895 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cb496-16b7-4b27-8c50-f5d94cc62b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:57.963796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cf2cb496-16b7-4b27-8c50-f5d94cc62b40 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:07:57.965116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cf2cb496-16b7-4b27-8c50-f5d94cc62b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:58.008135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:58.084195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:07:58.368509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37fa14-cf19-4f67-894a-5730e653e4a4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:07:58.370132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bb37fa14-cf19-4f67-894a-5730e653e4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:58.487948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37fa14-cf19-4f67-894a-5730e653e4a4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:07:58.533382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:07:58.572328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff655307-8ddb-49da-827d-09b07d563542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:07:58.572961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff655307-8ddb-49da-827d-09b07d563542 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:07:58.573235 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff655307-8ddb-49da-827d-09b07d563542 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:07:58.573498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff655307-8ddb-49da-827d-09b07d563542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:07:58.575100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ff655307-8ddb-49da-827d-09b07d563542 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:07:58.692120 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff655307-8ddb-49da-827d-09b07d563542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:07:58.693505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff655307-8ddb-49da-827d-09b07d563542 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:07:58.713207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 731/1464] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:08:00.902039 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d86c90-0394-4813-a5ac-c9a7290d3df3 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:00.902594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d86c90-0394-4813-a5ac-c9a7290d3df3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:00.902594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d86c90-0394-4813-a5ac-c9a7290d3df3 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:00.903861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:08:00.910828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b7301-3017-46a8-8974-c429facc811d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:00.911577 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b7301-3017-46a8-8974-c429facc811d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:00.911749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b7301-3017-46a8-8974-c429facc811d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:00.983614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f91b7301-3017-46a8-8974-c429facc811d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:08:00.984892 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f91b7301-3017-46a8-8974-c429facc811d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:01.010369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:01.118764 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:08:01.399509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d86c90-0394-4813-a5ac-c9a7290d3df3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:08:01.402044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a3d86c90-0394-4813-a5ac-c9a7290d3df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:01.531608 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d86c90-0394-4813-a5ac-c9a7290d3df3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:08:01.615944 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:01.623443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337c22d-87d0-4f78-b192-1d5f6009cfaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:08:01.624061 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c22d-87d0-4f78-b192-1d5f6009cfaf service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:01.624336 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c22d-87d0-4f78-b192-1d5f6009cfaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:01.624630 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c22d-87d0-4f78-b192-1d5f6009cfaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:01.626296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d337c22d-87d0-4f78-b192-1d5f6009cfaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:01.743188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337c22d-87d0-4f78-b192-1d5f6009cfaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:08:01.744717 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337c22d-87d0-4f78-b192-1d5f6009cfaf service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:08:01.762649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 735/1467] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:08:03.978920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3505f2-8ac1-4cd2-a57a-1f33dca13f98 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:03.979481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3505f2-8ac1-4cd2-a57a-1f33dca13f98 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:03.979481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3505f2-8ac1-4cd2-a57a-1f33dca13f98 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:03.980888 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:08:03.987227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc8d13-823f-4f2b-bd9b-461eca205ca1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:03.987666 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc8d13-823f-4f2b-bd9b-461eca205ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:03.987774 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc8d13-823f-4f2b-bd9b-461eca205ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:04.064242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-99fc8d13-823f-4f2b-bd9b-461eca205ca1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:08:04.065521 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-99fc8d13-823f-4f2b-bd9b-461eca205ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:04.115902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:04.165923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:08:04.450288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3505f2-8ac1-4cd2-a57a-1f33dca13f98 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:08:04.451838 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bb3505f2-8ac1-4cd2-a57a-1f33dca13f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:04.562791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3505f2-8ac1-4cd2-a57a-1f33dca13f98 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:08:04.618601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:04.647029 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261ec48-1d04-4030-a227-602e9ff2da90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:08:04.647635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261ec48-1d04-4030-a227-602e9ff2da90 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:04.647909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261ec48-1d04-4030-a227-602e9ff2da90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:04.648156 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261ec48-1d04-4030-a227-602e9ff2da90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:04.649978 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8261ec48-1d04-4030-a227-602e9ff2da90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:04.787372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8261ec48-1d04-4030-a227-602e9ff2da90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:08:04.788863 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8261ec48-1d04-4030-a227-602e9ff2da90 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:08:04.814809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 734/1470] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:08:14.164017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=22 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:14.810118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=66 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:24.858778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049d1a7-c164-416b-857e-6ef351e7bdb7 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:24.859370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049d1a7-c164-416b-857e-6ef351e7bdb7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:24.859370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049d1a7-c164-416b-857e-6ef351e7bdb7 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:24.860563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 737/1471] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:08:24.866268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c5c29-1f85-4962-9588-6d578bb9863b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:24.866712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c5c29-1f85-4962-9588-6d578bb9863b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:24.866839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c5c29-1f85-4962-9588-6d578bb9863b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:24.939868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-148c5c29-1f85-4962-9588-6d578bb9863b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:08:24.941179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-148c5c29-1f85-4962-9588-6d578bb9863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:25.043816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 735/1472] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:08:25.338060 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa704a2-241a-4721-8ad2-709239caff0d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:25.338346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa704a2-241a-4721-8ad2-709239caff0d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:25.338619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa704a2-241a-4721-8ad2-709239caff0d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:25.339968 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 738/1473] 10.222.0.79 () {54 vars in 961 bytes} [Tue Sep 6 21:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:08:25.345431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c8e8-2372-4bb7-9f15-9bb0fb45e8e6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:25.345708 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c8e8-2372-4bb7-9f15-9bb0fb45e8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:25.345974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c8e8-2372-4bb7-9f15-9bb0fb45e8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:25.426100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ae39c8e8-2372-4bb7-9f15-9bb0fb45e8e6 None None] MFA Rules not processed for user `a76937ab52e445c59b9eb9638fd4a753`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:08:25.427326 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ae39c8e8-2372-4bb7-9f15-9bb0fb45e8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:25.449617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:25.517306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 736/1474] 10.222.0.79 () {58 vars in 1058 bytes} [Tue Sep 6 21:08:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:08:25.525513 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa704a2-241a-4721-8ad2-709239caff0d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:08:25.527083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0fa704a2-241a-4721-8ad2-709239caff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:25.642053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa704a2-241a-4721-8ad2-709239caff0d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:08:25.720622 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbc8fb-b9be-426b-984b-169c6870fb62 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a76937ab52e445c59b9eb9638fd4a753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:08:25.721488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbc8fb-b9be-426b-984b-169c6870fb62 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:25.721823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbc8fb-b9be-426b-984b-169c6870fb62 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:25.722185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbc8fb-b9be-426b-984b-169c6870fb62 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:25.724783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b7dbc8fb-b9be-426b-984b-169c6870fb62 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:25.781858 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:25.847128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbc8fb-b9be-426b-984b-169c6870fb62 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:08:25.848567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbc8fb-b9be-426b-984b-169c6870fb62 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:08:25.868619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 739/1475] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:08:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:08:28.119636 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7177a-3201-47ee-bb81-04c6f2f89b53 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:28.120235 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7177a-3201-47ee-bb81-04c6f2f89b53 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:28.120235 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7177a-3201-47ee-bb81-04c6f2f89b53 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:28.121564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 737/1476] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:08:28.127863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189695c-3771-43ab-a913-9db32f635e6e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:28.128531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189695c-3771-43ab-a913-9db32f635e6e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:28.128811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189695c-3771-43ab-a913-9db32f635e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:28.206129 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a189695c-3771-43ab-a913-9db32f635e6e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:08:28.207439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a189695c-3771-43ab-a913-9db32f635e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:28.332449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 740/1477] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:08:28.602336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7177a-3201-47ee-bb81-04c6f2f89b53 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:08:28.603882 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-55e7177a-3201-47ee-bb81-04c6f2f89b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:28.740645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7177a-3201-47ee-bb81-04c6f2f89b53 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:08:28.797687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:28.858933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501bc77-adc5-483d-ac04-8b7557a7001f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:08:28.859463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501bc77-adc5-483d-ac04-8b7557a7001f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:28.859738 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501bc77-adc5-483d-ac04-8b7557a7001f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:28.860006 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501bc77-adc5-483d-ac04-8b7557a7001f admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:28.860649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501bc77-adc5-483d-ac04-8b7557a7001f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:08:28.862435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-d501bc77-adc5-483d-ac04-8b7557a7001f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:08:28.862435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 21:08:28.862435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: Sep 06 21:08:28.863572 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501bc77-adc5-483d-ac04-8b7557a7001f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:08:28.882778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 738/1478] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Sep 6 21:08:28 2022] GET /identity/v3/projects => generated 2553 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:08:31.051398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78257c11-92e0-4b69-a034-79bfbf8e647d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:31.051988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78257c11-92e0-4b69-a034-79bfbf8e647d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:31.051988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78257c11-92e0-4b69-a034-79bfbf8e647d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:31.054209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 741/1479] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:08:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:08:31.063509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb301bc3-c0bd-457c-93aa-4bf4c40e84a4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:31.063917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb301bc3-c0bd-457c-93aa-4bf4c40e84a4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:31.064122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb301bc3-c0bd-457c-93aa-4bf4c40e84a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:31.138238 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bb301bc3-c0bd-457c-93aa-4bf4c40e84a4 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:08:31.139482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bb301bc3-c0bd-457c-93aa-4bf4c40e84a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:31.258836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 739/1480] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:08:31.561939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78257c11-92e0-4b69-a034-79bfbf8e647d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:08:31.563977 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-78257c11-92e0-4b69-a034-79bfbf8e647d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:31.575974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:31.728683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78257c11-92e0-4b69-a034-79bfbf8e647d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:08:31.811534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:31.860310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7bbce-4c92-4b35-b579-f74291708edc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:08:31.860891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7bbce-4c92-4b35-b579-f74291708edc service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:31.861175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7bbce-4c92-4b35-b579-f74291708edc service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:31.861443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7bbce-4c92-4b35-b579-f74291708edc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:31.863077 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0bb7bbce-4c92-4b35-b579-f74291708edc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:31.960791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7bbce-4c92-4b35-b579-f74291708edc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:08:31.962471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7bbce-4c92-4b35-b579-f74291708edc service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:08:31.981329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 742/1481] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:08:34.374502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19f2a-11bd-4663-ac3c-554572611e89 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:34.375437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19f2a-11bd-4663-ac3c-554572611e89 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:34.375437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19f2a-11bd-4663-ac3c-554572611e89 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:34.377740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 740/1482] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:08:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:08:34.386620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe173dd-7e4d-47c6-8f04-c9155a1e0375 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:34.387094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe173dd-7e4d-47c6-8f04-c9155a1e0375 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:34.387200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe173dd-7e4d-47c6-8f04-c9155a1e0375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:34.461093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2fe173dd-7e4d-47c6-8f04-c9155a1e0375 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:08:34.462423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2fe173dd-7e4d-47c6-8f04-c9155a1e0375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:34.517929 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:34.576184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 743/1483] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:08:34.872842 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e19f2a-11bd-4663-ac3c-554572611e89 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:08:34.874386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a1e19f2a-11bd-4663-ac3c-554572611e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:34.898238 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:34.982385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e19f2a-11bd-4663-ac3c-554572611e89 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:08:35.057858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa36ca-b59c-4137-b92b-ac35ce74b076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:08:35.058444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa36ca-b59c-4137-b92b-ac35ce74b076 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:35.058719 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa36ca-b59c-4137-b92b-ac35ce74b076 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:35.058991 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa36ca-b59c-4137-b92b-ac35ce74b076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:35.060603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b3fa36ca-b59c-4137-b92b-ac35ce74b076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:35.138474 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:35.177736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa36ca-b59c-4137-b92b-ac35ce74b076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:08:35.179186 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa36ca-b59c-4137-b92b-ac35ce74b076 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:08:35.201420 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 741/1484] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:08:37.452391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1841a-f65a-4823-94e6-5850e586be89 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:37.453052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1841a-f65a-4823-94e6-5850e586be89 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:37.453052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1841a-f65a-4823-94e6-5850e586be89 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:37.454339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 744/1485] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:08:37.461264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae0db6-71a0-4470-b819-95837cecedab None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:37.461719 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae0db6-71a0-4470-b819-95837cecedab None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:37.461840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae0db6-71a0-4470-b819-95837cecedab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:37.538018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f6ae0db6-71a0-4470-b819-95837cecedab None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:08:37.539325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f6ae0db6-71a0-4470-b819-95837cecedab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:37.645303 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 742/1486] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:08:37.933166 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1841a-f65a-4823-94e6-5850e586be89 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:08:37.935321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-55a1841a-f65a-4823-94e6-5850e586be89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:38.051349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1841a-f65a-4823-94e6-5850e586be89 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:08:38.125002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:38.127237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903261f-19a1-47af-a6ac-f041a1132855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:08:38.127956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903261f-19a1-47af-a6ac-f041a1132855 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:38.128231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903261f-19a1-47af-a6ac-f041a1132855 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:38.128492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903261f-19a1-47af-a6ac-f041a1132855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:38.130538 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1903261f-19a1-47af-a6ac-f041a1132855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:38.250254 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903261f-19a1-47af-a6ac-f041a1132855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:08:38.251727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903261f-19a1-47af-a6ac-f041a1132855 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:08:38.271503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 745/1487] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:08:40.653786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b1ad-19c9-46fe-8d4b-9e860a675c4f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:40.654392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b1ad-19c9-46fe-8d4b-9e860a675c4f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:40.654392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b1ad-19c9-46fe-8d4b-9e860a675c4f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:40.656462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 743/1488] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:08:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:08:40.664703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5152b0f-f224-449a-bfec-bf342d1ba27d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:40.665281 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5152b0f-f224-449a-bfec-bf342d1ba27d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:40.665281 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5152b0f-f224-449a-bfec-bf342d1ba27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:40.733756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e5152b0f-f224-449a-bfec-bf342d1ba27d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:08:40.734974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e5152b0f-f224-449a-bfec-bf342d1ba27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:40.790769 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:40.836778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 746/1489] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:08:41.130438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123b1ad-19c9-46fe-8d4b-9e860a675c4f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:08:41.132990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a123b1ad-19c9-46fe-8d4b-9e860a675c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:41.177023 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:41.262361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123b1ad-19c9-46fe-8d4b-9e860a675c4f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:08:41.360834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959f871-3149-4077-8ff8-205ba085cf13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:08:41.361448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959f871-3149-4077-8ff8-205ba085cf13 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:41.361725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959f871-3149-4077-8ff8-205ba085cf13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:41.361997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959f871-3149-4077-8ff8-205ba085cf13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:41.363601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e959f871-3149-4077-8ff8-205ba085cf13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:41.456307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:41.489372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959f871-3149-4077-8ff8-205ba085cf13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:08:41.490865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959f871-3149-4077-8ff8-205ba085cf13 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:08:41.510391 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 744/1490] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:08:43.879212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5552a-2fe8-47ec-b060-de8619c3ace1 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:43.879793 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5552a-2fe8-47ec-b060-de8619c3ace1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:43.879793 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5552a-2fe8-47ec-b060-de8619c3ace1 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:43.881183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 747/1491] 10.222.0.79 () {54 vars in 927 bytes} [Tue Sep 6 21:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:08:43.887031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d9e8e-7ab7-4fd1-b201-49408bcf62a8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:43.887510 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d9e8e-7ab7-4fd1-b201-49408bcf62a8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:43.887607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d9e8e-7ab7-4fd1-b201-49408bcf62a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:43.959548 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6b2d9e8e-7ab7-4fd1-b201-49408bcf62a8 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:08:43.960807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6b2d9e8e-7ab7-4fd1-b201-49408bcf62a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:44.066639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 745/1492] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Sep 6 21:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:08:44.356058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5552a-2fe8-47ec-b060-de8619c3ace1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:08:44.357711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4de5552a-2fe8-47ec-b060-de8619c3ace1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:44.480071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5552a-2fe8-47ec-b060-de8619c3ace1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:08:44.569992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1aa024-e0a0-4d4f-b1ac-bc3325734500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:08:44.570610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1aa024-e0a0-4d4f-b1ac-bc3325734500 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:08:44.570894 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1aa024-e0a0-4d4f-b1ac-bc3325734500 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:08:44.571203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1aa024-e0a0-4d4f-b1ac-bc3325734500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:08:44.572896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0b1aa024-e0a0-4d4f-b1ac-bc3325734500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:08:44.583454 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:44.695401 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1aa024-e0a0-4d4f-b1ac-bc3325734500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:08:44.696845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1aa024-e0a0-4d4f-b1ac-bc3325734500 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:08:44.715733 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 748/1493] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:08:54.064593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=82 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:08:54.713315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=50 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:10:06.336915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b1e99-95fe-4d27-af5f-3ffe18ec0520 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:06.337591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b1e99-95fe-4d27-af5f-3ffe18ec0520 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:06.337591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b1e99-95fe-4d27-af5f-3ffe18ec0520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:06.440319 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-598b1e99-95fe-4d27-af5f-3ffe18ec0520 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:10:06.441585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-598b1e99-95fe-4d27-af5f-3ffe18ec0520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:06.545100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 746/1494] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:10:06.553652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec49763-0260-459c-9b6d-67b4588379e7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:06.554176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec49763-0260-459c-9b6d-67b4588379e7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:06.554176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec49763-0260-459c-9b6d-67b4588379e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:06.641512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1ec49763-0260-459c-9b6d-67b4588379e7 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:10:06.642869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1ec49763-0260-459c-9b6d-67b4588379e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:06.752281 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 749/1495] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:10:06.757210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b1e99-95fe-4d27-af5f-3ffe18ec0520 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:10:06.758500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-598b1e99-95fe-4d27-af5f-3ffe18ec0520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:06.770606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b1e99-95fe-4d27-af5f-3ffe18ec0520 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:10:06.834399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:10:06.868698 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c89aa8-6f68-494c-9af9-67a566366ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:10:06.869315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c89aa8-6f68-494c-9af9-67a566366ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:06.869587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c89aa8-6f68-494c-9af9-67a566366ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:06.869850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c89aa8-6f68-494c-9af9-67a566366ee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:06.870482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c89aa8-6f68-494c-9af9-67a566366ee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:10:06.870775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c89aa8-6f68-494c-9af9-67a566366ee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:10:06.871781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-c7c89aa8-6f68-494c-9af9-67a566366ee3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:10:06.871781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 21:10:06.871781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: Sep 06 21:10:06.872645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c89aa8-6f68-494c-9af9-67a566366ee3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:10:06.884856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 747/1496] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:10:06.893614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec49763-0260-459c-9b6d-67b4588379e7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:10:06.903115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec49763-0260-459c-9b6d-67b4588379e7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:10:06.975888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:10:07.012172 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa73c00-5a4a-4e6b-8c64-93de6a3c00bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:10:07.012826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa73c00-5a4a-4e6b-8c64-93de6a3c00bc admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:07.013091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa73c00-5a4a-4e6b-8c64-93de6a3c00bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:07.013349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa73c00-5a4a-4e6b-8c64-93de6a3c00bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:07.013985 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa73c00-5a4a-4e6b-8c64-93de6a3c00bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:10:07.015840 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-aaa73c00-5a4a-4e6b-8c64-93de6a3c00bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:10:07.015840 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 21:10:07.015840 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: Sep 06 21:10:07.017071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa73c00-5a4a-4e6b-8c64-93de6a3c00bc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:10:07.201773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 750/1497] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:10:06 2022] POST /identity/v3/projects => generated 377 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:10:07.206035 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c89aa8-6f68-494c-9af9-67a566366ee3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:10:07.207589 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c7c89aa8-6f68-494c-9af9-67a566366ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:07.241994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c89aa8-6f68-494c-9af9-67a566366ee3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:10:07.349858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81150fb4-e09d-4667-a5d3-0d131863b814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:10:07.350586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81150fb4-e09d-4667-a5d3-0d131863b814 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:07.350705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81150fb4-e09d-4667-a5d3-0d131863b814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:07.350973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81150fb4-e09d-4667-a5d3-0d131863b814 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:07.351595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81150fb4-e09d-4667-a5d3-0d131863b814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:10:07.352746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-81150fb4-e09d-4667-a5d3-0d131863b814 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:10:07.352746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 21:10:07.352746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: Sep 06 21:10:07.353947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81150fb4-e09d-4667-a5d3-0d131863b814 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:10:07.398500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 748/1498] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:10:07 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:10:07.402894 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa73c00-5a4a-4e6b-8c64-93de6a3c00bc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:10:07.404434 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-aaa73c00-5a4a-4e6b-8c64-93de6a3c00bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:07.471814 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa73c00-5a4a-4e6b-8c64-93de6a3c00bc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:10:07.563410 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:10:07.577825 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3c701-1774-4bc7-9480-1cce3cd8a681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:10:07.578437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3c701-1774-4bc7-9480-1cce3cd8a681 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:07.578709 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3c701-1774-4bc7-9480-1cce3cd8a681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:07.578970 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3c701-1774-4bc7-9480-1cce3cd8a681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:07.579615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3c701-1774-4bc7-9480-1cce3cd8a681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:10:07.580669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-64e3c701-1774-4bc7-9480-1cce3cd8a681 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:10:07.580669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 21:10:07.580669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: Sep 06 21:10:07.581627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3c701-1774-4bc7-9480-1cce3cd8a681 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:10:07.596395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 751/1499] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:10:07.600760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81150fb4-e09d-4667-a5d3-0d131863b814 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:10:07.602280 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-81150fb4-e09d-4667-a5d3-0d131863b814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:07.614197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81150fb4-e09d-4667-a5d3-0d131863b814 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:10:07.641724 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:10:07.718144 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c1fc3-0512-4cae-89e5-36f08476269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:10:07.718775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c1fc3-0512-4cae-89e5-36f08476269f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:07.719050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c1fc3-0512-4cae-89e5-36f08476269f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:07.719316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c1fc3-0512-4cae-89e5-36f08476269f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:07.719951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c1fc3-0512-4cae-89e5-36f08476269f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:10:07.721047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-df1c1fc3-0512-4cae-89e5-36f08476269f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:10:07.721047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 21:10:07.721047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: Sep 06 21:10:07.722072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c1fc3-0512-4cae-89e5-36f08476269f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:10:07.737226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 749/1500] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:10:07.741876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3c701-1774-4bc7-9480-1cce3cd8a681 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:10:07.743658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-64e3c701-1774-4bc7-9480-1cce3cd8a681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:07.781662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3c701-1774-4bc7-9480-1cce3cd8a681 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:10:07.887589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573889a4-489c-46e3-91fb-6f22cf25efcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:10:07.888230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573889a4-489c-46e3-91fb-6f22cf25efcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:07.888506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573889a4-489c-46e3-91fb-6f22cf25efcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:07.888810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573889a4-489c-46e3-91fb-6f22cf25efcc admin admin] PATH_INFO: `/v3/projects/d92df7fa48cc4ef989f1ddf0f3f041b3/users/9ca001db7a834605bd343a2406edacb5/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:07.915618 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573889a4-489c-46e3-91fb-6f22cf25efcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92df7fa48cc4ef989f1ddf0f3f041b3, user_id=9ca001db7a834605bd343a2406edacb5, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:10:07.916917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-573889a4-489c-46e3-91fb-6f22cf25efcc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:10:07.916917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 21:10:07.916917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: Sep 06 21:10:07.918998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573889a4-489c-46e3-91fb-6f22cf25efcc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:10:07.956569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 752/1501] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:10:07 2022] PUT /identity/v3/projects/d92df7fa48cc4ef989f1ddf0f3f041b3/users/9ca001db7a834605bd343a2406edacb5/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:10:07.969078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21952076-0b02-4848-9f12-e3664ada718d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:07.969377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21952076-0b02-4848-9f12-e3664ada718d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:07.969654 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21952076-0b02-4848-9f12-e3664ada718d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:08.045945 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-21952076-0b02-4848-9f12-e3664ada718d None None] MFA Rules not processed for user `9ca001db7a834605bd343a2406edacb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:10:08.047244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-21952076-0b02-4848-9f12-e3664ada718d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:08.135765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 750/1502] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:10:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:10:08.156453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6e03b-41be-4a02-b3c7-91add45058e9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:08.156761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6e03b-41be-4a02-b3c7-91add45058e9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:08.157023 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6e03b-41be-4a02-b3c7-91add45058e9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:08.158798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 753/1503] 10.222.0.79 () {54 vars in 964 bytes} [Tue Sep 6 21:10:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:10:08.164375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b80295-a39d-427f-b7cc-eb58c27f385a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:08.164811 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b80295-a39d-427f-b7cc-eb58c27f385a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:08.165086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b80295-a39d-427f-b7cc-eb58c27f385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:08.206652 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:10:08.271447 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-93b80295-a39d-427f-b7cc-eb58c27f385a None None] MFA Rules not processed for user `f50ae3b9997d4dc39c3d67ca40bbc566`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:10:08.272823 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-93b80295-a39d-427f-b7cc-eb58c27f385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:08.375543 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 751/1504] 10.222.0.79 () {58 vars in 1061 bytes} [Tue Sep 6 21:10:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:10:08.383586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6e03b-41be-4a02-b3c7-91add45058e9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:10:08.385155 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d6a6e03b-41be-4a02-b3c7-91add45058e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:08.440581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:10:08.504053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6e03b-41be-4a02-b3c7-91add45058e9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:10:08.591537 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbca9c-94c7-41c4-a18c-bf431b41b261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:10:08.592261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbca9c-94c7-41c4-a18c-bf431b41b261 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:08.592589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbca9c-94c7-41c4-a18c-bf431b41b261 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:08.592934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbca9c-94c7-41c4-a18c-bf431b41b261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:08.594996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-92dbca9c-94c7-41c4-a18c-bf431b41b261 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:08.711590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbca9c-94c7-41c4-a18c-bf431b41b261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:10:08.713021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbca9c-94c7-41c4-a18c-bf431b41b261 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:10:08.716611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:10:08.738097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 754/1505] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:10:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:10:08.793079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5a308-051b-4b52-b1c0-f37e58f85516 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:08.793346 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5a308-051b-4b52-b1c0-f37e58f85516 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:08.793638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5a308-051b-4b52-b1c0-f37e58f85516 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:08.794937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 752/1506] 10.222.0.79 () {52 vars in 830 bytes} [Tue Sep 6 21:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:10:08.798817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbca9c-94c7-41c4-a18c-bf431b41b261 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:10:08.800108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-92dbca9c-94c7-41c4-a18c-bf431b41b261 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:08.913531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbca9c-94c7-41c4-a18c-bf431b41b261 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:10:09.021947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:10:09.022605 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:09.022873 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:09.023139 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] PATH_INFO: `/v3/users/9ca001db7a834605bd343a2406edacb5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:09.036573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca001db7a834605bd343a2406edacb5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:10:09.037672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:10:09.037672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 21:10:09.037672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: Sep 06 21:10:09.038866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:10:09.048741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:10:09.109889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca001db7a834605bd343a2406edacb5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:10:09.110336 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca001db7a834605bd343a2406edacb5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:10:09.127332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca001db7a834605bd343a2406edacb5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:10:09.159684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca001db7a834605bd343a2406edacb5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:10:09.160624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca001db7a834605bd343a2406edacb5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:10:09.179427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 755/1507] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:10:08 2022] DELETE /identity/v3/users/9ca001db7a834605bd343a2406edacb5 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:10:09.184286 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2bdf4-2c68-4fb7-8ec5-250f8250600c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:09.184570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2bdf4-2c68-4fb7-8ec5-250f8250600c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:09.184868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2bdf4-2c68-4fb7-8ec5-250f8250600c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:09.243446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-55c2bdf4-2c68-4fb7-8ec5-250f8250600c None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:10:09.244683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-55c2bdf4-2c68-4fb7-8ec5-250f8250600c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:09.328865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:10:09.342770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 753/1508] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:10:09.353033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:10:09.354358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:09.456954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afa5b0-5901-4bc0-932a-4da974e16ce0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:10:09.511658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:10:09.534825 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a32a5a-e3e1-4768-a5ae-ccf5735cdf7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:10:09.535483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a32a5a-e3e1-4768-a5ae-ccf5735cdf7f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:09.535754 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a32a5a-e3e1-4768-a5ae-ccf5735cdf7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:09.536021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a32a5a-e3e1-4768-a5ae-ccf5735cdf7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:09.537666 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-18a32a5a-e3e1-4768-a5ae-ccf5735cdf7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:09.652902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a32a5a-e3e1-4768-a5ae-ccf5735cdf7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:10:09.654473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a32a5a-e3e1-4768-a5ae-ccf5735cdf7f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:10:09.674267 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 756/1509] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:10:10.030831 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2bdf4-2c68-4fb7-8ec5-250f8250600c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:10:10.032430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-55c2bdf4-2c68-4fb7-8ec5-250f8250600c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:10:10.045578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2bdf4-2c68-4fb7-8ec5-250f8250600c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:10:10.169634 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:10:10.170239 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:10:10.170514 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:10:10.170780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98 admin admin] PATH_INFO: `/v3/projects/d92df7fa48cc4ef989f1ddf0f3f041b3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:10:10.181054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d92df7fa48cc4ef989f1ddf0f3f041b3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:10:10.182084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:10:10.182084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 21:10:10.182084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: Sep 06 21:10:10.183297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:10:10.243464 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd92df7fa48cc4ef989f1ddf0f3f041b3', 'request_id': 'req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:10:10.253360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd92df7fa48cc4ef989f1ddf0f3f041b3', 'request_id': 'req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:10:10.267406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd92df7fa48cc4ef989f1ddf0f3f041b3', 'request_id': 'req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:10:10.293566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98 admin admin] The token cache is being invalidate because project d92df7fa48cc4ef989f1ddf0f3f041b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:10:10.293851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:10:10.294381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8b5bd78a-83eb-4f9c-a0c7-5d1f9a692e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:10:10.295743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 754/1510] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:10:10 2022] DELETE /identity/v3/projects/d92df7fa48cc4ef989f1ddf0f3f041b3 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:10:19.671841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=62 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:10:20.291450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:11:44.994451 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76e87e-cb75-49cd-8b82-31f19a25d28e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:44.995224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76e87e-cb75-49cd-8b82-31f19a25d28e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:44.995224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76e87e-cb75-49cd-8b82-31f19a25d28e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:44.996995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 21:11:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:11:45.004088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de10b83-512b-45e6-a1b4-241c9f486b53 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:45.004787 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de10b83-512b-45e6-a1b4-241c9f486b53 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:45.004961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de10b83-512b-45e6-a1b4-241c9f486b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:45.098299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8de10b83-512b-45e6-a1b4-241c9f486b53 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:11:45.099580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8de10b83-512b-45e6-a1b4-241c9f486b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:11:45.222125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:11:45.373642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76e87e-cb75-49cd-8b82-31f19a25d28e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:11:45.376113 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6d76e87e-cb75-49cd-8b82-31f19a25d28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:11:45.501807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76e87e-cb75-49cd-8b82-31f19a25d28e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:11:45.508058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269319ad-6981-4f0e-8dff-18f4d06eac16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:11:45.508679 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269319ad-6981-4f0e-8dff-18f4d06eac16 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:45.508974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269319ad-6981-4f0e-8dff-18f4d06eac16 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:45.509255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269319ad-6981-4f0e-8dff-18f4d06eac16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:45.517638 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269319ad-6981-4f0e-8dff-18f4d06eac16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:11:45.519227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269319ad-6981-4f0e-8dff-18f4d06eac16 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:11:45.528701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 758/1513] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:11:46.633002 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd5c95-1bca-4dac-a940-6f163a8d80ed None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:46.633571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd5c95-1bca-4dac-a940-6f163a8d80ed None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:46.633571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd5c95-1bca-4dac-a940-6f163a8d80ed None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:46.635041 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 21:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:11:46.642653 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b7ab4-00a7-47fe-8da1-f10626c13a81 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:46.643347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b7ab4-00a7-47fe-8da1-f10626c13a81 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:46.643463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b7ab4-00a7-47fe-8da1-f10626c13a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:46.675080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-df1b7ab4-00a7-47fe-8da1-f10626c13a81 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:11:46.676696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-df1b7ab4-00a7-47fe-8da1-f10626c13a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:11:46.686251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 21:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:11:46.843519 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd5c95-1bca-4dac-a940-6f163a8d80ed None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:11:46.865765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd5c95-1bca-4dac-a940-6f163a8d80ed None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:11:46.874034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37034f1d-cb9e-4514-8502-92d89408b905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:11:46.874669 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37034f1d-cb9e-4514-8502-92d89408b905 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:46.874938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37034f1d-cb9e-4514-8502-92d89408b905 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:46.875201 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37034f1d-cb9e-4514-8502-92d89408b905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:46.884376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37034f1d-cb9e-4514-8502-92d89408b905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:11:46.886004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37034f1d-cb9e-4514-8502-92d89408b905 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:11:46.896838 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 757/1516] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:11:47.430491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb32085-92ed-424f-8434-38dd790e4fab None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:47.430767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb32085-92ed-424f-8434-38dd790e4fab None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:47.431024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb32085-92ed-424f-8434-38dd790e4fab None None] PATH_INFO: `/v3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:47.432312 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 760/1517] 10.222.0.73 () {54 vars in 925 bytes} [Tue Sep 6 21:11:47 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:11:47.439515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e170aa-c903-4e0e-9701-518beaf5b4ea None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:47.439798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e170aa-c903-4e0e-9701-518beaf5b4ea None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:47.440071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e170aa-c903-4e0e-9701-518beaf5b4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:47.503258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e4e170aa-c903-4e0e-9701-518beaf5b4ea None None] MFA Rules not processed for user `c40b5ff21d1940749bf479cc936fc0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:11:47.504487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e4e170aa-c903-4e0e-9701-518beaf5b4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:11:47.521982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:11:47.584435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 758/1518] 10.222.0.73 () {58 vars in 1015 bytes} [Tue Sep 6 21:11:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:11:56.668626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=56 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:11:57.126534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b9bd4-6a75-4550-a497-aae960e2eae0 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:57.127364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b9bd4-6a75-4550-a497-aae960e2eae0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:57.127364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b9bd4-6a75-4550-a497-aae960e2eae0 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:57.128615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 761/1519] 10.222.0.79 () {54 vars in 961 bytes} [Tue Sep 6 21:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:11:57.134430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313f55d-cdf3-4a10-af38-e7eb58cd2662 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:57.134922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313f55d-cdf3-4a10-af38-e7eb58cd2662 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:57.135015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313f55d-cdf3-4a10-af38-e7eb58cd2662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:57.156943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a313f55d-cdf3-4a10-af38-e7eb58cd2662 None None] MFA Rules not processed for user `c40b5ff21d1940749bf479cc936fc0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:11:57.158182 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a313f55d-cdf3-4a10-af38-e7eb58cd2662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:11:57.168063 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 759/1520] 10.222.0.79 () {58 vars in 1058 bytes} [Tue Sep 6 21:11:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:11:57.175760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b9bd4-6a75-4550-a497-aae960e2eae0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:11:57.192231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b9bd4-6a75-4550-a497-aae960e2eae0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:11:57.197264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f5f57-7bca-4f3c-b292-3447cb033873 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:11:57.197873 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f5f57-7bca-4f3c-b292-3447cb033873 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:57.198150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f5f57-7bca-4f3c-b292-3447cb033873 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:57.198419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f5f57-7bca-4f3c-b292-3447cb033873 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:57.206960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f5f57-7bca-4f3c-b292-3447cb033873 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:11:57.208546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f5f57-7bca-4f3c-b292-3447cb033873 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:11:57.219177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 762/1521] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:11:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:11:58.400954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658be75e-768f-4dfe-9d4d-76ba2ff2209e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:58.401820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658be75e-768f-4dfe-9d4d-76ba2ff2209e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:58.401820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658be75e-768f-4dfe-9d4d-76ba2ff2209e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:58.403984 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 21:11:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:11:58.411316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dae59-b124-47d7-b849-3aca8d7e8c8f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:58.412125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dae59-b124-47d7-b849-3aca8d7e8c8f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:58.412274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dae59-b124-47d7-b849-3aca8d7e8c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:58.439728 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e25dae59-b124-47d7-b849-3aca8d7e8c8f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:11:58.440972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e25dae59-b124-47d7-b849-3aca8d7e8c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:11:58.450025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 21:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:11:58.609167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658be75e-768f-4dfe-9d4d-76ba2ff2209e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:11:58.629013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658be75e-768f-4dfe-9d4d-76ba2ff2209e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:11:58.636024 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7244b-377e-46ea-966f-a5bc7cc87129 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:11:58.636647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7244b-377e-46ea-966f-a5bc7cc87129 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:58.636956 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7244b-377e-46ea-966f-a5bc7cc87129 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:58.637231 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7244b-377e-46ea-966f-a5bc7cc87129 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:58.646759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7244b-377e-46ea-966f-a5bc7cc87129 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:11:58.648370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7244b-377e-46ea-966f-a5bc7cc87129 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:11:58.658820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 761/1524] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:11:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:11:59.738665 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362660d3-5d61-4977-a1e9-a1e70bea6acf None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:59.739249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362660d3-5d61-4977-a1e9-a1e70bea6acf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:59.739249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362660d3-5d61-4977-a1e9-a1e70bea6acf None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:59.740456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 21:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:11:59.747996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374cf2e-45d1-4808-a549-af51388316b2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:59.748935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374cf2e-45d1-4808-a549-af51388316b2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:59.748935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374cf2e-45d1-4808-a549-af51388316b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:59.777908 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a374cf2e-45d1-4808-a549-af51388316b2 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:11:59.779101 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a374cf2e-45d1-4808-a549-af51388316b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:11:59.788672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:11:59.942474 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362660d3-5d61-4977-a1e9-a1e70bea6acf None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:11:59.960059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362660d3-5d61-4977-a1e9-a1e70bea6acf None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:11:59.966384 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987bec5d-80ac-468a-a4ba-d660f5cd1a24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:11:59.966979 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bec5d-80ac-468a-a4ba-d660f5cd1a24 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:11:59.967264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bec5d-80ac-468a-a4ba-d660f5cd1a24 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:11:59.967614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bec5d-80ac-468a-a4ba-d660f5cd1a24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:11:59.976277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987bec5d-80ac-468a-a4ba-d660f5cd1a24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:11:59.978036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987bec5d-80ac-468a-a4ba-d660f5cd1a24 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:11:59.987498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 765/1527] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:12:04.001427 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a2aa7-3ce5-4cca-9766-590089cfde7b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:12:04.002731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a2aa7-3ce5-4cca-9766-590089cfde7b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:12:04.002731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a2aa7-3ce5-4cca-9766-590089cfde7b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:12:04.004673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:12:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:12:04.011094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55268465-2c52-45d0-bb91-aae830610135 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:12:04.011646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55268465-2c52-45d0-bb91-aae830610135 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:12:04.011646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55268465-2c52-45d0-bb91-aae830610135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:12:04.032079 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-55268465-2c52-45d0-bb91-aae830610135 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:12:04.033331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-55268465-2c52-45d0-bb91-aae830610135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:12:04.042087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:12:04.338169 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a2aa7-3ce5-4cca-9766-590089cfde7b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:12:04.340576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1c4a2aa7-3ce5-4cca-9766-590089cfde7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:12:04.436150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a2aa7-3ce5-4cca-9766-590089cfde7b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:12:04.439958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab1673-9c3b-4536-852a-57908a646af6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:12:04.440592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab1673-9c3b-4536-852a-57908a646af6 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:12:04.440864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab1673-9c3b-4536-852a-57908a646af6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:12:04.441157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab1673-9c3b-4536-852a-57908a646af6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:12:04.450554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab1673-9c3b-4536-852a-57908a646af6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:12:04.451990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab1673-9c3b-4536-852a-57908a646af6 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:12:04.461783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 764/1530] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:12:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:12:06.615413 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e47d1-1ed2-4059-b45b-18dcc5b79671 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:12:06.615999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e47d1-1ed2-4059-b45b-18dcc5b79671 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:12:06.615999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e47d1-1ed2-4059-b45b-18dcc5b79671 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:12:06.617375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:12:06.622955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c10c7c-e5df-43af-83bf-98832bac038d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:12:06.623423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c10c7c-e5df-43af-83bf-98832bac038d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:12:06.623529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c10c7c-e5df-43af-83bf-98832bac038d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:12:06.647055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-84c10c7c-e5df-43af-83bf-98832bac038d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:12:06.648913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-84c10c7c-e5df-43af-83bf-98832bac038d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:12:06.657926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:12:06.946009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e47d1-1ed2-4059-b45b-18dcc5b79671 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:12:06.948196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e20e47d1-1ed2-4059-b45b-18dcc5b79671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:12:06.966484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e47d1-1ed2-4059-b45b-18dcc5b79671 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:12:06.972219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d878ef6-f186-48be-a84d-00e4ab177ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:12:06.972988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878ef6-f186-48be-a84d-00e4ab177ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:12:06.973271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878ef6-f186-48be-a84d-00e4ab177ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:12:06.973546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878ef6-f186-48be-a84d-00e4ab177ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:12:06.984240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d878ef6-f186-48be-a84d-00e4ab177ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:12:06.985722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d878ef6-f186-48be-a84d-00e4ab177ff4 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:12:06.996559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 768/1533] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:12:10.757599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac69a97-25d0-45c2-b0f3-7cb909ed64cb None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:12:10.758188 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac69a97-25d0-45c2-b0f3-7cb909ed64cb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:12:10.758188 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac69a97-25d0-45c2-b0f3-7cb909ed64cb None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:12:10.759732 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:12:10.765591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1fbb3-88a6-41dc-9ff5-c74eb36b34bc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:12:10.766230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1fbb3-88a6-41dc-9ff5-c74eb36b34bc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:12:10.766230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1fbb3-88a6-41dc-9ff5-c74eb36b34bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:12:10.786842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9ca1fbb3-88a6-41dc-9ff5-c74eb36b34bc None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:12:10.788065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9ca1fbb3-88a6-41dc-9ff5-c74eb36b34bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:12:10.796745 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:12:11.100873 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac69a97-25d0-45c2-b0f3-7cb909ed64cb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:12:11.102645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0ac69a97-25d0-45c2-b0f3-7cb909ed64cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:12:11.171554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:12:11.189294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac69a97-25d0-45c2-b0f3-7cb909ed64cb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:12:11.193162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6a01b-3e07-460b-a74d-51b2274794a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:12:11.193777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6a01b-3e07-460b-a74d-51b2274794a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:12:11.194078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6a01b-3e07-460b-a74d-51b2274794a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:12:11.194346 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6a01b-3e07-460b-a74d-51b2274794a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:12:11.204648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6a01b-3e07-460b-a74d-51b2274794a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:12:11.206119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6a01b-3e07-460b-a74d-51b2274794a2 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:12:11.216844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 767/1536] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:12:13.430791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246844f-5177-48a6-a21a-c735a29fd779 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:12:13.431451 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246844f-5177-48a6-a21a-c735a29fd779 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:12:13.431451 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246844f-5177-48a6-a21a-c735a29fd779 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:12:13.432843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 21:12:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:12:13.438612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201b7b3-eeae-4ea4-8491-ae76e5443d51 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:12:13.439058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201b7b3-eeae-4ea4-8491-ae76e5443d51 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:12:13.439148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201b7b3-eeae-4ea4-8491-ae76e5443d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:12:13.463599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f201b7b3-eeae-4ea4-8491-ae76e5443d51 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:12:13.464693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f201b7b3-eeae-4ea4-8491-ae76e5443d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:12:13.474009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 21:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:12:13.779466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246844f-5177-48a6-a21a-c735a29fd779 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:12:13.781589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f246844f-5177-48a6-a21a-c735a29fd779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:12:13.796486 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246844f-5177-48a6-a21a-c735a29fd779 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:12:13.800408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78533d6a-a875-4e54-8683-f0f11d12c6b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:12:13.801395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78533d6a-a875-4e54-8683-f0f11d12c6b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:12:13.801671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78533d6a-a875-4e54-8683-f0f11d12c6b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:12:13.801933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78533d6a-a875-4e54-8683-f0f11d12c6b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:12:13.811621 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78533d6a-a875-4e54-8683-f0f11d12c6b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:12:13.813106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78533d6a-a875-4e54-8683-f0f11d12c6b2 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:12:13.823166 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 771/1539] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:12:23.455693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=11 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:12:23.818589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=33 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:38.880474 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ac5df-5039-4e5c-86c2-8414f9b9c108 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:38.881151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ac5df-5039-4e5c-86c2-8414f9b9c108 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:38.881151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ac5df-5039-4e5c-86c2-8414f9b9c108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:38.902742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fd1a6-1788-4c32-834e-94e8f93ccd38 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:38.904788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fd1a6-1788-4c32-834e-94e8f93ccd38 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:38.904788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fd1a6-1788-4c32-834e-94e8f93ccd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:38.913751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-886ac5df-5039-4e5c-86c2-8414f9b9c108 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:38.914996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-886ac5df-5039-4e5c-86c2-8414f9b9c108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:38.934457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 769/1540] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:13:38.938909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-494fd1a6-1788-4c32-834e-94e8f93ccd38 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:38.940170 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-494fd1a6-1788-4c32-834e-94e8f93ccd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:38.946520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470b782-dee6-4a07-9d93-c6e35d53cd94 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:38.946913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470b782-dee6-4a07-9d93-c6e35d53cd94 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:38.947104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470b782-dee6-4a07-9d93-c6e35d53cd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:38.958790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 772/1541] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:13:38.969149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e987e-5001-4d22-b1c3-e236a3bb2e3a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:38.969497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e987e-5001-4d22-b1c3-e236a3bb2e3a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:38.969697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e987e-5001-4d22-b1c3-e236a3bb2e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:38.995632 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5470b782-dee6-4a07-9d93-c6e35d53cd94 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.002673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5470b782-dee6-4a07-9d93-c6e35d53cd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.019794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 770/1542] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:13:39.024846 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470b782-dee6-4a07-9d93-c6e35d53cd94 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.029224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d92e987e-5001-4d22-b1c3-e236a3bb2e3a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.030466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d92e987e-5001-4d22-b1c3-e236a3bb2e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.043157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470b782-dee6-4a07-9d93-c6e35d53cd94 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.046808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 773/1543] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:13:39.050780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf6ddb-e626-4528-89d2-4366fae535ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.051414 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf6ddb-e626-4528-89d2-4366fae535ea admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.051691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf6ddb-e626-4528-89d2-4366fae535ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.051962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf6ddb-e626-4528-89d2-4366fae535ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.052622 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf6ddb-e626-4528-89d2-4366fae535ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:13:39.052947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf6ddb-e626-4528-89d2-4366fae535ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.054581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf6ddb-e626-4528-89d2-4366fae535ea admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.065002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e987e-5001-4d22-b1c3-e236a3bb2e3a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.071684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 771/1544] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:13:39.080533 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e987e-5001-4d22-b1c3-e236a3bb2e3a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.088977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf6ddb-e626-4528-89d2-4366fae535ea admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.089826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28a625-21bc-4b3a-a7f4-d9d9dffa21d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.090411 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28a625-21bc-4b3a-a7f4-d9d9dffa21d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.090691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28a625-21bc-4b3a-a7f4-d9d9dffa21d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.090962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28a625-21bc-4b3a-a7f4-d9d9dffa21d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.091623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28a625-21bc-4b3a-a7f4-d9d9dffa21d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:13:39.091922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28a625-21bc-4b3a-a7f4-d9d9dffa21d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.092992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-fe28a625-21bc-4b3a-a7f4-d9d9dffa21d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:13:39.092992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 21:13:39.092992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: Sep 06 21:13:39.093881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28a625-21bc-4b3a-a7f4-d9d9dffa21d5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.102267 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf6ddb-e626-4528-89d2-4366fae535ea admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.112616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34e748-41df-4519-a2f6-74714ef4af55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.113268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34e748-41df-4519-a2f6-74714ef4af55 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.113551 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34e748-41df-4519-a2f6-74714ef4af55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.113824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34e748-41df-4519-a2f6-74714ef4af55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.114469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34e748-41df-4519-a2f6-74714ef4af55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.115509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-3b34e748-41df-4519-a2f6-74714ef4af55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:13:39.115509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 21:13:39.115509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: Sep 06 21:13:39.116603 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 774/1545] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:13:39.116755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34e748-41df-4519-a2f6-74714ef4af55 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.140824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28a625-21bc-4b3a-a7f4-d9d9dffa21d5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.163604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28a625-21bc-4b3a-a7f4-d9d9dffa21d5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.174031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077841ec-594c-4a7d-896a-9c1227848b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.174776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077841ec-594c-4a7d-896a-9c1227848b91 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.175464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077841ec-594c-4a7d-896a-9c1227848b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.175790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077841ec-594c-4a7d-896a-9c1227848b91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.176503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077841ec-594c-4a7d-896a-9c1227848b91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.181148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077841ec-594c-4a7d-896a-9c1227848b91 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.190832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 772/1546] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:39.200524 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34e748-41df-4519-a2f6-74714ef4af55 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.246066 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 775/1547] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:39.259123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e56bfe-1f69-47bc-a649-552f34a3c762 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.259460 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e56bfe-1f69-47bc-a649-552f34a3c762 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.259720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e56bfe-1f69-47bc-a649-552f34a3c762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.301965 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-11e56bfe-1f69-47bc-a649-552f34a3c762 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.302372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34e748-41df-4519-a2f6-74714ef4af55 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.303524 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-11e56bfe-1f69-47bc-a649-552f34a3c762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.382800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 776/1548] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:39.383727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e56bfe-1f69-47bc-a649-552f34a3c762 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.390588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5cdc5-5ddc-4bf2-a8a1-20ea367c2f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.392025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5cdc5-5ddc-4bf2-a8a1-20ea367c2f58 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.392255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5cdc5-5ddc-4bf2-a8a1-20ea367c2f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.392519 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5cdc5-5ddc-4bf2-a8a1-20ea367c2f58 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.393195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5cdc5-5ddc-4bf2-a8a1-20ea367c2f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.395678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5cdc5-5ddc-4bf2-a8a1-20ea367c2f58 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.396012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e56bfe-1f69-47bc-a649-552f34a3c762 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.410554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e862579-1e34-429d-bda9-480430371723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.411240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e862579-1e34-429d-bda9-480430371723 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.411524 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e862579-1e34-429d-bda9-480430371723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.411790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e862579-1e34-429d-bda9-480430371723 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.412560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e862579-1e34-429d-bda9-480430371723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.413729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-8e862579-1e34-429d-bda9-480430371723 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:13:39.413729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 21:13:39.413729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: Sep 06 21:13:39.415052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e862579-1e34-429d-bda9-480430371723 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.424258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 773/1549] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:13:39.426271 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f002d1-3293-4b79-a02f-e0627afe8467 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.426555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f002d1-3293-4b79-a02f-e0627afe8467 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.427168 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f002d1-3293-4b79-a02f-e0627afe8467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.440154 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 777/1550] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/users => generated 293 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:39.441615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4a7a8-0028-4815-b7eb-bb5dee55134c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.441898 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4a7a8-0028-4815-b7eb-bb5dee55134c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.442168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4a7a8-0028-4815-b7eb-bb5dee55134c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.449539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-47f002d1-3293-4b79-a02f-e0627afe8467 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.450748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-47f002d1-3293-4b79-a02f-e0627afe8467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.459260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 774/1551] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:39.460472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c30ae7-42a3-4399-8e92-b404612614c0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.460838 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c30ae7-42a3-4399-8e92-b404612614c0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.461049 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c30ae7-42a3-4399-8e92-b404612614c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.461854 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-76a4a7a8-0028-4815-b7eb-bb5dee55134c None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.463410 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-76a4a7a8-0028-4815-b7eb-bb5dee55134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.471504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 778/1552] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:39.472933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db70180-3acc-49ed-91bb-c6813a32ae2d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.473275 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db70180-3acc-49ed-91bb-c6813a32ae2d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.473604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db70180-3acc-49ed-91bb-c6813a32ae2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.481088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b2c30ae7-42a3-4399-8e92-b404612614c0 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.482294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b2c30ae7-42a3-4399-8e92-b404612614c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.490415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 775/1553] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:39.491615 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b040f8-a510-4f02-86ac-3dd9b6348c17 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.491949 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b040f8-a510-4f02-86ac-3dd9b6348c17 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.492224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b040f8-a510-4f02-86ac-3dd9b6348c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.493696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2db70180-3acc-49ed-91bb-c6813a32ae2d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.494932 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2db70180-3acc-49ed-91bb-c6813a32ae2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.504240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 779/1554] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:39.505474 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa76250-7e67-4d0c-8471-63bfc666ca08 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.505758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa76250-7e67-4d0c-8471-63bfc666ca08 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.506028 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa76250-7e67-4d0c-8471-63bfc666ca08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.514342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-06b040f8-a510-4f02-86ac-3dd9b6348c17 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.515655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-06b040f8-a510-4f02-86ac-3dd9b6348c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.525781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 776/1555] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:39.525911 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1fa76250-7e67-4d0c-8471-63bfc666ca08 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.526579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b040f8-a510-4f02-86ac-3dd9b6348c17 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.527137 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1fa76250-7e67-4d0c-8471-63bfc666ca08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.535445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 780/1556] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:39.536258 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa76250-7e67-4d0c-8471-63bfc666ca08 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.538359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b040f8-a510-4f02-86ac-3dd9b6348c17 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.543385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77c36e-5485-4d63-a550-b10c868fc438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.543995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77c36e-5485-4d63-a550-b10c868fc438 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.544270 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77c36e-5485-4d63-a550-b10c868fc438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.544530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77c36e-5485-4d63-a550-b10c868fc438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.545201 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77c36e-5485-4d63-a550-b10c868fc438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.545885 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa76250-7e67-4d0c-8471-63bfc666ca08 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.546853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77c36e-5485-4d63-a550-b10c868fc438 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.551089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13b85e-9af6-4b78-8540-81f94c0cd2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.551696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13b85e-9af6-4b78-8540-81f94c0cd2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.551960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13b85e-9af6-4b78-8540-81f94c0cd2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.552224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13b85e-9af6-4b78-8540-81f94c0cd2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.552883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13b85e-9af6-4b78-8540-81f94c0cd2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.554479 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13b85e-9af6-4b78-8540-81f94c0cd2e6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.561433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 777/1557] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:39.561433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76327ba3-b0f6-4ac0-b5e6-a9b2aad471a2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.561433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76327ba3-b0f6-4ac0-b5e6-a9b2aad471a2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.561433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76327ba3-b0f6-4ac0-b5e6-a9b2aad471a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.565087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 781/1558] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:39.566390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767e7a1-76f1-4bf3-b321-aa673e54610f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.566673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767e7a1-76f1-4bf3-b321-aa673e54610f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.566944 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767e7a1-76f1-4bf3-b321-aa673e54610f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.580003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-76327ba3-b0f6-4ac0-b5e6-a9b2aad471a2 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.581317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-76327ba3-b0f6-4ac0-b5e6-a9b2aad471a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.592577 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 778/1559] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:39.593173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b767e7a1-76f1-4bf3-b321-aa673e54610f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.594013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dabac-13ea-4299-9baa-ac5723d31dd6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.594297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dabac-13ea-4299-9baa-ac5723d31dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.594399 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b767e7a1-76f1-4bf3-b321-aa673e54610f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.594561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dabac-13ea-4299-9baa-ac5723d31dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.602680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:39.603386 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 782/1560] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:39.604048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767e7a1-76f1-4bf3-b321-aa673e54610f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.614549 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767e7a1-76f1-4bf3-b321-aa673e54610f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.619250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b487f0-0687-4d62-a045-337945aa346d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.619849 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b487f0-0687-4d62-a045-337945aa346d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.620259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b487f0-0687-4d62-a045-337945aa346d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.620540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b487f0-0687-4d62-a045-337945aa346d admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.621242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4f5dabac-13ea-4299-9baa-ac5723d31dd6 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.621367 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b487f0-0687-4d62-a045-337945aa346d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:13:39.621547 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b487f0-0687-4d62-a045-337945aa346d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.622973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4f5dabac-13ea-4299-9baa-ac5723d31dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.623305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b487f0-0687-4d62-a045-337945aa346d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.631693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 779/1561] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:39.634161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdbfe6-4919-49e6-98e4-8e340f769a9a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.634582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdbfe6-4919-49e6-98e4-8e340f769a9a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.634858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdbfe6-4919-49e6-98e4-8e340f769a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.638995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 783/1562] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:13:39.640589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9c539-bc83-4455-bdd0-32d5caa72831 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.641092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9c539-bc83-4455-bdd0-32d5caa72831 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.641779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9c539-bc83-4455-bdd0-32d5caa72831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.657207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-43cdbfe6-4919-49e6-98e4-8e340f769a9a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.658344 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-43cdbfe6-4919-49e6-98e4-8e340f769a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.661808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0bd9c539-bc83-4455-bdd0-32d5caa72831 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:39.663028 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0bd9c539-bc83-4455-bdd0-32d5caa72831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:39.666757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 780/1563] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:39.667580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdbfe6-4919-49e6-98e4-8e340f769a9a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.673655 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 784/1564] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:39.674453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9c539-bc83-4455-bdd0-32d5caa72831 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.679893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdbfe6-4919-49e6-98e4-8e340f769a9a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.685369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446135a-f969-4ae7-b97a-d3a15d390145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.685970 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446135a-f969-4ae7-b97a-d3a15d390145 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.686247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446135a-f969-4ae7-b97a-d3a15d390145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.686511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9c539-bc83-4455-bdd0-32d5caa72831 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.686625 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446135a-f969-4ae7-b97a-d3a15d390145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.687150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446135a-f969-4ae7-b97a-d3a15d390145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.688793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446135a-f969-4ae7-b97a-d3a15d390145 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.691812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d69506-193c-43ba-a066-0ee33824fe55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.692425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d69506-193c-43ba-a066-0ee33824fe55 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.692730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d69506-193c-43ba-a066-0ee33824fe55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.693020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d69506-193c-43ba-a066-0ee33824fe55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.693831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d69506-193c-43ba-a066-0ee33824fe55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.695462 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d69506-193c-43ba-a066-0ee33824fe55 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.699525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 781/1565] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:39.700611 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446135a-f969-4ae7-b97a-d3a15d390145 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.707918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 785/1566] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:39.708777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d69506-193c-43ba-a066-0ee33824fe55 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.715476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446135a-f969-4ae7-b97a-d3a15d390145 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.718562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d69506-193c-43ba-a066-0ee33824fe55 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.721462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a013fca-3bba-4927-abd5-ece9cea1a5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.722175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a013fca-3bba-4927-abd5-ece9cea1a5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.722452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a013fca-3bba-4927-abd5-ece9cea1a5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.722741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a013fca-3bba-4927-abd5-ece9cea1a5c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.723288 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09a438-80af-4e28-abb9-919986d45f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.723428 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a013fca-3bba-4927-abd5-ece9cea1a5c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:13:39.723787 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a013fca-3bba-4927-abd5-ece9cea1a5c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.723907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09a438-80af-4e28-abb9-919986d45f98 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.724143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09a438-80af-4e28-abb9-919986d45f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.724416 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09a438-80af-4e28-abb9-919986d45f98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.725088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09a438-80af-4e28-abb9-919986d45f98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:13:39.725389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09a438-80af-4e28-abb9-919986d45f98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.725505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a013fca-3bba-4927-abd5-ece9cea1a5c5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.726997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09a438-80af-4e28-abb9-919986d45f98 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.738715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 782/1567] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:13:39.739616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a013fca-3bba-4927-abd5-ece9cea1a5c5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.747271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 786/1568] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:13:39.747271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09a438-80af-4e28-abb9-919986d45f98 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.751936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09a438-80af-4e28-abb9-919986d45f98 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.756163 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a013fca-3bba-4927-abd5-ece9cea1a5c5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.756787 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32a5f1-c100-4a22-9c86-7f4b50122aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.757394 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32a5f1-c100-4a22-9c86-7f4b50122aac admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.757676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32a5f1-c100-4a22-9c86-7f4b50122aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.757943 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32a5f1-c100-4a22-9c86-7f4b50122aac admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.758571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32a5f1-c100-4a22-9c86-7f4b50122aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.760490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32a5f1-c100-4a22-9c86-7f4b50122aac admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.760685 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed15be-4b8b-4582-9865-1e7c5ddced93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.761278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed15be-4b8b-4582-9865-1e7c5ddced93 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.761557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed15be-4b8b-4582-9865-1e7c5ddced93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.761854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed15be-4b8b-4582-9865-1e7c5ddced93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.762479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed15be-4b8b-4582-9865-1e7c5ddced93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:13:39.762984 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed15be-4b8b-4582-9865-1e7c5ddced93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.765332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed15be-4b8b-4582-9865-1e7c5ddced93 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.772535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:39.777577 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 783/1569] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:13:39.778534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed15be-4b8b-4582-9865-1e7c5ddced93 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.790707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed15be-4b8b-4582-9865-1e7c5ddced93 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.794743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 787/1570] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:39.794743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32a5f1-c100-4a22-9c86-7f4b50122aac admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.853443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e80301-f3a7-468c-8eb7-8bcec231cf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.854180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e80301-f3a7-468c-8eb7-8bcec231cf13 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.854558 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e80301-f3a7-468c-8eb7-8bcec231cf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.854867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e80301-f3a7-468c-8eb7-8bcec231cf13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.855581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e80301-f3a7-468c-8eb7-8bcec231cf13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:13:39.855926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32a5f1-c100-4a22-9c86-7f4b50122aac admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.856028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e80301-f3a7-468c-8eb7-8bcec231cf13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.857722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e80301-f3a7-468c-8eb7-8bcec231cf13 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.860896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139524db-bb15-4e0c-bd52-094a612e5dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.861560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139524db-bb15-4e0c-bd52-094a612e5dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.861841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139524db-bb15-4e0c-bd52-094a612e5dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.862110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139524db-bb15-4e0c-bd52-094a612e5dc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.862756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139524db-bb15-4e0c-bd52-094a612e5dc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:13:39.863051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139524db-bb15-4e0c-bd52-094a612e5dc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.864649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139524db-bb15-4e0c-bd52-094a612e5dc9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.869799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 784/1571] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:13:39.870595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e80301-f3a7-468c-8eb7-8bcec231cf13 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.884547 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 788/1572] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:13:39.884547 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139524db-bb15-4e0c-bd52-094a612e5dc9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.884881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e80301-f3a7-468c-8eb7-8bcec231cf13 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.887652 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cfcb4-4b26-4b0d-b2c2-a1368b2c47ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.888316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cfcb4-4b26-4b0d-b2c2-a1368b2c47ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.888645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cfcb4-4b26-4b0d-b2c2-a1368b2c47ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.888954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cfcb4-4b26-4b0d-b2c2-a1368b2c47ea admin admin] PATH_INFO: `/v3/projects/2e39683d687e424fbed4539cb4040060/users/54fe3d99f05c45f099804fccb4679503/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.889440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139524db-bb15-4e0c-bd52-094a612e5dc9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.894977 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94608a0e-97d0-4bcc-8467-7dbff25c8dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:39.895661 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94608a0e-97d0-4bcc-8467-7dbff25c8dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:39.895943 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94608a0e-97d0-4bcc-8467-7dbff25c8dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:39.896212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94608a0e-97d0-4bcc-8467-7dbff25c8dd5 admin admin] PATH_INFO: `/v3/projects/8ba0f5f714694c249c785253abd18dc8/users/8bf3e32edd3a41daaecfe6c73a761363/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:39.904096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cfcb4-4b26-4b0d-b2c2-a1368b2c47ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e39683d687e424fbed4539cb4040060, user_id=54fe3d99f05c45f099804fccb4679503, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.905180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-ba9cfcb4-4b26-4b0d-b2c2-a1368b2c47ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:13:39.905180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 21:13:39.905180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: Sep 06 21:13:39.907105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cfcb4-4b26-4b0d-b2c2-a1368b2c47ea admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.911980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94608a0e-97d0-4bcc-8467-7dbff25c8dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba0f5f714694c249c785253abd18dc8, user_id=8bf3e32edd3a41daaecfe6c73a761363, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:39.914975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94608a0e-97d0-4bcc-8467-7dbff25c8dd5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:39.922632 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 785/1573] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:39 2022] PUT /identity/v3/projects/2e39683d687e424fbed4539cb4040060/users/54fe3d99f05c45f099804fccb4679503/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:39.923546 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cfcb4-4b26-4b0d-b2c2-a1368b2c47ea admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.933277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 789/1574] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:39 2022] PUT /identity/v3/projects/8ba0f5f714694c249c785253abd18dc8/users/8bf3e32edd3a41daaecfe6c73a761363/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:39.934260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94608a0e-97d0-4bcc-8467-7dbff25c8dd5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:39.998797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94608a0e-97d0-4bcc-8467-7dbff25c8dd5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:39.999196 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cfcb4-4b26-4b0d-b2c2-a1368b2c47ea admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.003983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572409d-4fce-47c2-ad20-dd4eee202c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.004505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e58aa-afbf-4c6f-b452-ee2eed78e3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.004680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572409d-4fce-47c2-ad20-dd4eee202c78 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.004984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572409d-4fce-47c2-ad20-dd4eee202c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.005144 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e58aa-afbf-4c6f-b452-ee2eed78e3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.005240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572409d-4fce-47c2-ad20-dd4eee202c78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.005382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e58aa-afbf-4c6f-b452-ee2eed78e3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.005682 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e58aa-afbf-4c6f-b452-ee2eed78e3b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.005856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572409d-4fce-47c2-ad20-dd4eee202c78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.006314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e58aa-afbf-4c6f-b452-ee2eed78e3b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.007861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572409d-4fce-47c2-ad20-dd4eee202c78 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.008418 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e58aa-afbf-4c6f-b452-ee2eed78e3b9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.035705 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 790/1575] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:40.036809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572409d-4fce-47c2-ad20-dd4eee202c78 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.037585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 786/1576] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:13:39 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:40.038375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e58aa-afbf-4c6f-b452-ee2eed78e3b9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.051576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:40.095085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:40.110026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e58aa-afbf-4c6f-b452-ee2eed78e3b9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.110465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572409d-4fce-47c2-ad20-dd4eee202c78 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.114828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0be2d-80af-47f5-83b3-a27b948fc07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.115140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044010c-f666-450b-831a-7eeb5436b21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.115468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0be2d-80af-47f5-83b3-a27b948fc07e admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.115745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0be2d-80af-47f5-83b3-a27b948fc07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.115853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044010c-f666-450b-831a-7eeb5436b21a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.116019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0be2d-80af-47f5-83b3-a27b948fc07e admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.116127 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044010c-f666-450b-831a-7eeb5436b21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.116379 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044010c-f666-450b-831a-7eeb5436b21a admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.116666 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0be2d-80af-47f5-83b3-a27b948fc07e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.117045 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044010c-f666-450b-831a-7eeb5436b21a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.118656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0be2d-80af-47f5-83b3-a27b948fc07e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.118946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044010c-f666-450b-831a-7eeb5436b21a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.141399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 787/1577] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:13:40 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:40.142362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0be2d-80af-47f5-83b3-a27b948fc07e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.151019 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 791/1578] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:13:40 2022] POST /identity/v3/projects => generated 401 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:40.151868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044010c-f666-450b-831a-7eeb5436b21a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.152033 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0be2d-80af-47f5-83b3-a27b948fc07e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.210258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75385fdd-4433-4ee6-813a-44e891b55050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.210796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75385fdd-4433-4ee6-813a-44e891b55050 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.211078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75385fdd-4433-4ee6-813a-44e891b55050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.211346 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75385fdd-4433-4ee6-813a-44e891b55050 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.211987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75385fdd-4433-4ee6-813a-44e891b55050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.213993 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75385fdd-4433-4ee6-813a-44e891b55050 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.216599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044010c-f666-450b-831a-7eeb5436b21a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.221847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d41b86-6d58-4630-bef3-512dfe359ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.222564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d41b86-6d58-4630-bef3-512dfe359ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.222934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d41b86-6d58-4630-bef3-512dfe359ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.223292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d41b86-6d58-4630-bef3-512dfe359ea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.224092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d41b86-6d58-4630-bef3-512dfe359ea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.226142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d41b86-6d58-4630-bef3-512dfe359ea5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.245978 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 788/1579] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:13:40 2022] POST /identity/v3/projects => generated 391 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:40.245978 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477158c-324e-45ae-a546-0349bcd28af3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.245978 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477158c-324e-45ae-a546-0349bcd28af3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.245978 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477158c-324e-45ae-a546-0349bcd28af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.256554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 792/1580] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:13:40 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:40.256554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9627cb-5e08-4698-bc5c-473423c8db3e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.256554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9627cb-5e08-4698-bc5c-473423c8db3e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.256554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9627cb-5e08-4698-bc5c-473423c8db3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.264186 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8477158c-324e-45ae-a546-0349bcd28af3 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:40.265428 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8477158c-324e-45ae-a546-0349bcd28af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:40.285602 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0d9627cb-5e08-4698-bc5c-473423c8db3e None None] MFA Rules not processed for user `8bf3e32edd3a41daaecfe6c73a761363`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:40.286927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0d9627cb-5e08-4698-bc5c-473423c8db3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:40.330557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 789/1581] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:40.331551 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477158c-324e-45ae-a546-0349bcd28af3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.341576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477158c-324e-45ae-a546-0349bcd28af3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.346096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656ba6e-e5c3-45e2-a570-194a607d60af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.346731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656ba6e-e5c3-45e2-a570-194a607d60af admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.347026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656ba6e-e5c3-45e2-a570-194a607d60af admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.347277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656ba6e-e5c3-45e2-a570-194a607d60af admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.347930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656ba6e-e5c3-45e2-a570-194a607d60af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.350123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656ba6e-e5c3-45e2-a570-194a607d60af admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.351161 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 793/1582] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:40.351161 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9627cb-5e08-4698-bc5c-473423c8db3e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.364540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9627cb-5e08-4698-bc5c-473423c8db3e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.370369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5155b76-96fc-435e-869b-0a194c2f1299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.370522 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5155b76-96fc-435e-869b-0a194c2f1299 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.370694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5155b76-96fc-435e-869b-0a194c2f1299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.371089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5155b76-96fc-435e-869b-0a194c2f1299 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.372047 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5155b76-96fc-435e-869b-0a194c2f1299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.374151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5155b76-96fc-435e-869b-0a194c2f1299 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.376826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 790/1583] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:13:40 2022] POST /identity/v3/users => generated 285 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:40.378243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656ba6e-e5c3-45e2-a570-194a607d60af admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.386368 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:40.390816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656ba6e-e5c3-45e2-a570-194a607d60af admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.397190 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c189bd5-1207-4d68-8b7b-f7eef8d9753d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.397821 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c189bd5-1207-4d68-8b7b-f7eef8d9753d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.398080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c189bd5-1207-4d68-8b7b-f7eef8d9753d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.398312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c189bd5-1207-4d68-8b7b-f7eef8d9753d admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.399024 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c189bd5-1207-4d68-8b7b-f7eef8d9753d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.404042 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 794/1584] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:13:40 2022] POST /identity/v3/users => generated 289 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:40.404236 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c189bd5-1207-4d68-8b7b-f7eef8d9753d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.404350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5155b76-96fc-435e-869b-0a194c2f1299 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.413370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5155b76-96fc-435e-869b-0a194c2f1299 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.417367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 791/1585] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:40.417367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c189bd5-1207-4d68-8b7b-f7eef8d9753d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.418224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bc3cf-bd04-4168-9dc9-c4a005dd7418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.418837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bc3cf-bd04-4168-9dc9-c4a005dd7418 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.419357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bc3cf-bd04-4168-9dc9-c4a005dd7418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.419637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bc3cf-bd04-4168-9dc9-c4a005dd7418 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.420275 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bc3cf-bd04-4168-9dc9-c4a005dd7418 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.422254 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bc3cf-bd04-4168-9dc9-c4a005dd7418 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.424953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c189bd5-1207-4d68-8b7b-f7eef8d9753d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.429763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68233e-dd86-412c-bf46-25177dddc370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.430332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68233e-dd86-412c-bf46-25177dddc370 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.430605 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68233e-dd86-412c-bf46-25177dddc370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.430869 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68233e-dd86-412c-bf46-25177dddc370 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.431482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68233e-dd86-412c-bf46-25177dddc370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.433401 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68233e-dd86-412c-bf46-25177dddc370 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.445969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 795/1586] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:13:40 2022] POST /identity/v3/users => generated 301 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:40.447032 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bc3cf-bd04-4168-9dc9-c4a005dd7418 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.459395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 792/1587] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:13:40 2022] POST /identity/v3/users => generated 296 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:40.459395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68233e-dd86-412c-bf46-25177dddc370 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.459692 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bc3cf-bd04-4168-9dc9-c4a005dd7418 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.463669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5d528-6611-42d5-be4a-5f50f8eaa2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.464210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5d528-6611-42d5-be4a-5f50f8eaa2ae admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.464495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5d528-6611-42d5-be4a-5f50f8eaa2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.464790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5d528-6611-42d5-be4a-5f50f8eaa2ae admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.465425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5d528-6611-42d5-be4a-5f50f8eaa2ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.467628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5d528-6611-42d5-be4a-5f50f8eaa2ae admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.479997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:40.494689 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 796/1588] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:13:40 2022] POST /identity/v3/users => generated 287 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:40.495315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5d528-6611-42d5-be4a-5f50f8eaa2ae admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.505572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5d528-6611-42d5-be4a-5f50f8eaa2ae admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.510569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725423d-4914-4ed9-85da-0128972f02b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.511792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725423d-4914-4ed9-85da-0128972f02b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.512075 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725423d-4914-4ed9-85da-0128972f02b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.512344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725423d-4914-4ed9-85da-0128972f02b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.513251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725423d-4914-4ed9-85da-0128972f02b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.514874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725423d-4914-4ed9-85da-0128972f02b1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.514988 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68233e-dd86-412c-bf46-25177dddc370 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.518557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04eca9-36dc-4fa3-8d69-39303d65d131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.519164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04eca9-36dc-4fa3-8d69-39303d65d131 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.519444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04eca9-36dc-4fa3-8d69-39303d65d131 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.519714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04eca9-36dc-4fa3-8d69-39303d65d131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.525341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 797/1589] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:40.526627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9cdb5-2aee-4e37-8dde-f37e3e6c0fd7 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.526969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9cdb5-2aee-4e37-8dde-f37e3e6c0fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.527248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9cdb5-2aee-4e37-8dde-f37e3e6c0fd7 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.528607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 798/1590] 10.222.0.79 () {54 vars in 962 bytes} [Tue Sep 6 21:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:13:40.530599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9cdb5-2aee-4e37-8dde-f37e3e6c0fd7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.531913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04eca9-36dc-4fa3-8d69-39303d65d131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.533514 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04eca9-36dc-4fa3-8d69-39303d65d131 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.543708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9cdb5-2aee-4e37-8dde-f37e3e6c0fd7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.544165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 793/1591] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:40.545015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04eca9-36dc-4fa3-8d69-39303d65d131 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.548280 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8b6c2-5539-4739-810e-0fb86c15b3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.548918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8b6c2-5539-4739-810e-0fb86c15b3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.549191 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8b6c2-5539-4739-810e-0fb86c15b3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.549481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8b6c2-5539-4739-810e-0fb86c15b3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.550652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8b6c2-5539-4739-810e-0fb86c15b3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.552263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8b6c2-5539-4739-810e-0fb86c15b3a1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.555210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04eca9-36dc-4fa3-8d69-39303d65d131 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.560129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da7046-d563-4fad-bfa9-5ea7fa51c4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.561036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da7046-d563-4fad-bfa9-5ea7fa51c4cb admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.561387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da7046-d563-4fad-bfa9-5ea7fa51c4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.561729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da7046-d563-4fad-bfa9-5ea7fa51c4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.562448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da7046-d563-4fad-bfa9-5ea7fa51c4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.566321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 799/1592] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:40.566483 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da7046-d563-4fad-bfa9-5ea7fa51c4cb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.566594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8b6c2-5539-4739-810e-0fb86c15b3a1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.576366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8b6c2-5539-4739-810e-0fb86c15b3a1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.580074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 794/1593] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:40.580074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da7046-d563-4fad-bfa9-5ea7fa51c4cb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.590807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da7046-d563-4fad-bfa9-5ea7fa51c4cb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.591614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729e595-0610-4860-a0e4-e0505cbc1ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.592378 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729e595-0610-4860-a0e4-e0505cbc1ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.592609 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729e595-0610-4860-a0e4-e0505cbc1ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.592905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729e595-0610-4860-a0e4-e0505cbc1ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.593612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729e595-0610-4860-a0e4-e0505cbc1ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.595243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729e595-0610-4860-a0e4-e0505cbc1ca9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.596071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf41737-f1d9-4e0a-b57d-a6eddfc94ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.596840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf41737-f1d9-4e0a-b57d-a6eddfc94ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.597093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf41737-f1d9-4e0a-b57d-a6eddfc94ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.597373 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf41737-f1d9-4e0a-b57d-a6eddfc94ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.598089 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf41737-f1d9-4e0a-b57d-a6eddfc94ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.599809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf41737-f1d9-4e0a-b57d-a6eddfc94ca6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.608438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 800/1594] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:40.610906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 795/1595] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:40.614208 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf41737-f1d9-4e0a-b57d-a6eddfc94ca6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.614393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729e595-0610-4860-a0e4-e0505cbc1ca9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.624265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf41737-f1d9-4e0a-b57d-a6eddfc94ca6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.626331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729e595-0610-4860-a0e4-e0505cbc1ca9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.630057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333545db-b3ca-49da-a05d-379c7c796b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.630784 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333545db-b3ca-49da-a05d-379c7c796b31 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.631306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333545db-b3ca-49da-a05d-379c7c796b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.631688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333545db-b3ca-49da-a05d-379c7c796b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.631802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976b2a9-1824-4508-ba2e-691806aadeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.632499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976b2a9-1824-4508-ba2e-691806aadeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.632673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333545db-b3ca-49da-a05d-379c7c796b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.633036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976b2a9-1824-4508-ba2e-691806aadeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.633365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976b2a9-1824-4508-ba2e-691806aadeb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.634157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976b2a9-1824-4508-ba2e-691806aadeb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.634283 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333545db-b3ca-49da-a05d-379c7c796b31 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.636188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976b2a9-1824-4508-ba2e-691806aadeb6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.653176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 796/1596] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:40.653408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 801/1597] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:40.653521 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b9350-22bc-4304-b818-0e499a04e7be None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.653625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976b2a9-1824-4508-ba2e-691806aadeb6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.653725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b9350-22bc-4304-b818-0e499a04e7be None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.653725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b9350-22bc-4304-b818-0e499a04e7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.664559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976b2a9-1824-4508-ba2e-691806aadeb6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.669690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13378801-119d-4e08-9015-2f8c3ec09e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.670301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13378801-119d-4e08-9015-2f8c3ec09e0a admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.670583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13378801-119d-4e08-9015-2f8c3ec09e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.670850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13378801-119d-4e08-9015-2f8c3ec09e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.671504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13378801-119d-4e08-9015-2f8c3ec09e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.673421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13378801-119d-4e08-9015-2f8c3ec09e0a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.686473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 802/1598] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:40.687420 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13378801-119d-4e08-9015-2f8c3ec09e0a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.698308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9f2b9350-22bc-4304-b818-0e499a04e7be None None] MFA Rules not processed for user `12d51454d20c46d7a5e6d963a2f987cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:40.698992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13378801-119d-4e08-9015-2f8c3ec09e0a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.699796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9f2b9350-22bc-4304-b818-0e499a04e7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:40.704308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eec274-d9b2-4cb9-8e9e-1d239e341fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.705018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec274-d9b2-4cb9-8e9e-1d239e341fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.705293 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec274-d9b2-4cb9-8e9e-1d239e341fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.705594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec274-d9b2-4cb9-8e9e-1d239e341fc3 admin admin] PATH_INFO: `/v3/projects/a2ec2d845bf247a6b463e59e513dd936/users/c914e6a41ccc48f49194d57551bdf0d4/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.723585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eec274-d9b2-4cb9-8e9e-1d239e341fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ec2d845bf247a6b463e59e513dd936, user_id=c914e6a41ccc48f49194d57551bdf0d4, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.726722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eec274-d9b2-4cb9-8e9e-1d239e341fc3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.747635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 803/1599] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:40 2022] PUT /identity/v3/projects/a2ec2d845bf247a6b463e59e513dd936/users/c914e6a41ccc48f49194d57551bdf0d4/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:40.749723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eec274-d9b2-4cb9-8e9e-1d239e341fc3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.792620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 797/1600] 10.222.0.79 () {58 vars in 1059 bytes} [Tue Sep 6 21:13:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:40.801781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b9350-22bc-4304-b818-0e499a04e7be None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.814539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b9350-22bc-4304-b818-0e499a04e7be None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.829828 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eec274-d9b2-4cb9-8e9e-1d239e341fc3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.835280 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e39bf2-2e7c-42fc-9e6e-1698eb1849d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.838730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e39bf2-2e7c-42fc-9e6e-1698eb1849d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.838730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e39bf2-2e7c-42fc-9e6e-1698eb1849d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.838730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e39bf2-2e7c-42fc-9e6e-1698eb1849d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.838730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e39bf2-2e7c-42fc-9e6e-1698eb1849d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.841025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e39bf2-2e7c-42fc-9e6e-1698eb1849d4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.844825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:40.857120 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 804/1601] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:40.862596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e39bf2-2e7c-42fc-9e6e-1698eb1849d4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.875813 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e39bf2-2e7c-42fc-9e6e-1698eb1849d4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:40.881993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17369f84-e47b-49b7-9ba8-793eeb43696e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.882671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17369f84-e47b-49b7-9ba8-793eeb43696e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.882945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17369f84-e47b-49b7-9ba8-793eeb43696e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.883228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17369f84-e47b-49b7-9ba8-793eeb43696e admin admin] PATH_INFO: `/v3/projects/252880d276d2460395cc844fb42531cc/users/992a7f66b2f74e71a6803bcfcce86ab5/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.904153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17369f84-e47b-49b7-9ba8-793eeb43696e admin admin] RBAC: Authorizing `identity:create_grant(project_id=252880d276d2460395cc844fb42531cc, user_id=992a7f66b2f74e71a6803bcfcce86ab5, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.907320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17369f84-e47b-49b7-9ba8-793eeb43696e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.908293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4876c-ce10-46cc-ba8d-0bef0ed2c0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:40.908960 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4876c-ce10-46cc-ba8d-0bef0ed2c0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:40.909263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4876c-ce10-46cc-ba8d-0bef0ed2c0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:40.909547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4876c-ce10-46cc-ba8d-0bef0ed2c0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:40.910179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4876c-ce10-46cc-ba8d-0bef0ed2c0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:40.911778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4876c-ce10-46cc-ba8d-0bef0ed2c0e8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:40.927483 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 798/1602] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:40.927699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 805/1603] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:40 2022] PUT /identity/v3/projects/252880d276d2460395cc844fb42531cc/users/992a7f66b2f74e71a6803bcfcce86ab5/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:40.927699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17369f84-e47b-49b7-9ba8-793eeb43696e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.929230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4876c-ce10-46cc-ba8d-0bef0ed2c0e8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:40.949239 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4876c-ce10-46cc-ba8d-0bef0ed2c0e8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:41.003275 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:41.021363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17369f84-e47b-49b7-9ba8-793eeb43696e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:41.026623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91628e5-6362-42ad-bb88-fdfb8eaf3172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:41.027127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b42b71-aa41-4974-a69a-dd6e123d3c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:41.027497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91628e5-6362-42ad-bb88-fdfb8eaf3172 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.027842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91628e5-6362-42ad-bb88-fdfb8eaf3172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.027947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b42b71-aa41-4974-a69a-dd6e123d3c66 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.028243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b42b71-aa41-4974-a69a-dd6e123d3c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.028343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91628e5-6362-42ad-bb88-fdfb8eaf3172 admin admin] PATH_INFO: `/v3/projects/cabcd82424984a0ba003e5324b505e54/users/64033668b9fd4ba48eef09bdc3fdf2ee/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.028871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b42b71-aa41-4974-a69a-dd6e123d3c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.029586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b42b71-aa41-4974-a69a-dd6e123d3c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:41.031272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b42b71-aa41-4974-a69a-dd6e123d3c66 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:41.044834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 799/1604] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:41.045366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27678123-5708-4f81-a5c6-69d42a43e9f1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.045617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27678123-5708-4f81-a5c6-69d42a43e9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.045870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27678123-5708-4f81-a5c6-69d42a43e9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.048256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91628e5-6362-42ad-bb88-fdfb8eaf3172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cabcd82424984a0ba003e5324b505e54, user_id=64033668b9fd4ba48eef09bdc3fdf2ee, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:41.051427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91628e5-6362-42ad-bb88-fdfb8eaf3172 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:41.066803 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 806/1605] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:40 2022] PUT /identity/v3/projects/cabcd82424984a0ba003e5324b505e54/users/64033668b9fd4ba48eef09bdc3fdf2ee/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:41.068024 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-27678123-5708-4f81-a5c6-69d42a43e9f1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:41.068220 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91628e5-6362-42ad-bb88-fdfb8eaf3172 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:41.069492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-27678123-5708-4f81-a5c6-69d42a43e9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:41.137618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 800/1606] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:41.138507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27678123-5708-4f81-a5c6-69d42a43e9f1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:41.146501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91628e5-6362-42ad-bb88-fdfb8eaf3172 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:41.148417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27678123-5708-4f81-a5c6-69d42a43e9f1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:41.154468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f431731-6e62-4dbd-acdc-ad82fdf96496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:41.155148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f431731-6e62-4dbd-acdc-ad82fdf96496 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.155429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f431731-6e62-4dbd-acdc-ad82fdf96496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.155707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f431731-6e62-4dbd-acdc-ad82fdf96496 admin admin] PATH_INFO: `/v3/projects/341f27c705cf4f05b7de4a6c001eb661/users/ae3562403ce64a25b206d9a9fdbb6673/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.157715 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf291e-381f-4809-b2a8-07e0b0e27199 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:41.158510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf291e-381f-4809-b2a8-07e0b0e27199 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.158945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf291e-381f-4809-b2a8-07e0b0e27199 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.159278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf291e-381f-4809-b2a8-07e0b0e27199 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.173089 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f431731-6e62-4dbd-acdc-ad82fdf96496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=341f27c705cf4f05b7de4a6c001eb661, user_id=ae3562403ce64a25b206d9a9fdbb6673, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:41.176435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f431731-6e62-4dbd-acdc-ad82fdf96496 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:41.192951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 801/1607] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:41 2022] PUT /identity/v3/projects/341f27c705cf4f05b7de4a6c001eb661/users/ae3562403ce64a25b206d9a9fdbb6673/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:41.197335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f431731-6e62-4dbd-acdc-ad82fdf96496 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:41.236407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:41.246437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf291e-381f-4809-b2a8-07e0b0e27199 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:41.253923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf291e-381f-4809-b2a8-07e0b0e27199 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:41.266897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 807/1608] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:41.268331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dfa28-cf16-47c0-bf39-793274ed1ec4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.268881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dfa28-cf16-47c0-bf39-793274ed1ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.269356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dfa28-cf16-47c0-bf39-793274ed1ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.295780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f431731-6e62-4dbd-acdc-ad82fdf96496 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:41.308873 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c87061-337e-447b-8a99-3c8407860932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:41.309604 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c87061-337e-447b-8a99-3c8407860932 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.309916 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c87061-337e-447b-8a99-3c8407860932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.310561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c87061-337e-447b-8a99-3c8407860932 admin admin] PATH_INFO: `/v3/projects/ff397d6b3f4a42bf8b68b3bc55600dba/users/2ec47d6e2e9b40a6bd207c084041f7a9/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.314933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fa2dfa28-cf16-47c0-bf39-793274ed1ec4 None None] MFA Rules not processed for user `992a7f66b2f74e71a6803bcfcce86ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:41.316924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fa2dfa28-cf16-47c0-bf39-793274ed1ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:41.354095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c87061-337e-447b-8a99-3c8407860932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff397d6b3f4a42bf8b68b3bc55600dba, user_id=2ec47d6e2e9b40a6bd207c084041f7a9, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:41.356824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c87061-337e-447b-8a99-3c8407860932 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:41.381923 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 802/1609] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:41 2022] PUT /identity/v3/projects/ff397d6b3f4a42bf8b68b3bc55600dba/users/2ec47d6e2e9b40a6bd207c084041f7a9/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:41.387722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c87061-337e-447b-8a99-3c8407860932 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:41.391842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:41.408176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 808/1610] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:41.410899 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c244c6-373a-4c25-af03-2f0ddf2769e0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.411575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c244c6-373a-4c25-af03-2f0ddf2769e0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.412078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c244c6-373a-4c25-af03-2f0ddf2769e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.436716 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-65c244c6-373a-4c25-af03-2f0ddf2769e0 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:41.439181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-65c244c6-373a-4c25-af03-2f0ddf2769e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:41.495648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c87061-337e-447b-8a99-3c8407860932 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:41.501688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a910b23-c4d1-409c-9ae1-dcfeba83aa31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:41.502374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a910b23-c4d1-409c-9ae1-dcfeba83aa31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.502651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a910b23-c4d1-409c-9ae1-dcfeba83aa31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.502980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a910b23-c4d1-409c-9ae1-dcfeba83aa31 admin admin] PATH_INFO: `/v3/projects/8e2d0b17f13b4989bbb230a60d6c5662/users/8f011304deec4c798bff42fbb8cf4c09/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.522354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a910b23-c4d1-409c-9ae1-dcfeba83aa31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e2d0b17f13b4989bbb230a60d6c5662, user_id=8f011304deec4c798bff42fbb8cf4c09, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:41.525600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a910b23-c4d1-409c-9ae1-dcfeba83aa31 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:41.544806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 803/1611] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:41 2022] PUT /identity/v3/projects/8e2d0b17f13b4989bbb230a60d6c5662/users/8f011304deec4c798bff42fbb8cf4c09/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:41.545848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a910b23-c4d1-409c-9ae1-dcfeba83aa31 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:41.570959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 809/1612] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:41.570959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea928765-f4f4-4f16-a55f-364613874319 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.570959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea928765-f4f4-4f16-a55f-364613874319 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.570959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea928765-f4f4-4f16-a55f-364613874319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.623071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ea928765-f4f4-4f16-a55f-364613874319 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:41.624834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ea928765-f4f4-4f16-a55f-364613874319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:41.636515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a910b23-c4d1-409c-9ae1-dcfeba83aa31 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:41.640861 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc64bd-5d8a-4ddf-a080-cb0cf32c071b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:41.641479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc64bd-5d8a-4ddf-a080-cb0cf32c071b service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.641756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc64bd-5d8a-4ddf-a080-cb0cf32c071b service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.642058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc64bd-5d8a-4ddf-a080-cb0cf32c071b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.698037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 810/1613] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:41.700458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003ffbb-a957-4476-8a23-c49b414d6751 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.701616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003ffbb-a957-4476-8a23-c49b414d6751 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.702244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003ffbb-a957-4476-8a23-c49b414d6751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.726572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6003ffbb-a957-4476-8a23-c49b414d6751 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:41.728117 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:41.728541 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6003ffbb-a957-4476-8a23-c49b414d6751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:41.743677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 811/1614] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:41.748039 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003ffbb-a957-4476-8a23-c49b414d6751 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:41.752331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc64bd-5d8a-4ddf-a080-cb0cf32c071b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:41.753817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc64bd-5d8a-4ddf-a080-cb0cf32c071b service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:41.760939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003ffbb-a957-4476-8a23-c49b414d6751 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:41.765132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 804/1615] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:41.766355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3e7aa-595e-4f31-b3d5-207853694e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:41.766882 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9c515-05f3-4f78-824d-b4d524e9c0ee None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.767014 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3e7aa-595e-4f31-b3d5-207853694e49 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.767234 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9c515-05f3-4f78-824d-b4d524e9c0ee None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.767398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3e7aa-595e-4f31-b3d5-207853694e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.767624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9c515-05f3-4f78-824d-b4d524e9c0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.767739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3e7aa-595e-4f31-b3d5-207853694e49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.768336 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3e7aa-595e-4f31-b3d5-207853694e49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:41.770815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3e7aa-595e-4f31-b3d5-207853694e49 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:41.803477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-04a9c515-05f3-4f78-824d-b4d524e9c0ee None None] MFA Rules not processed for user `8f011304deec4c798bff42fbb8cf4c09`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:41.804906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-04a9c515-05f3-4f78-824d-b4d524e9c0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:41.831586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 812/1616] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:13:41 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:41.831586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3e7aa-595e-4f31-b3d5-207853694e49 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:41.905732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3e7aa-595e-4f31-b3d5-207853694e49 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:41.910477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a16160-7bf2-4d15-a5bb-285d83726586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:41.915760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a16160-7bf2-4d15-a5bb-285d83726586 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.915760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a16160-7bf2-4d15-a5bb-285d83726586 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.915760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a16160-7bf2-4d15-a5bb-285d83726586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.916079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 805/1617] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:41.916079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a21580-ab04-4041-82f0-c77f82949579 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.916079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a21580-ab04-4041-82f0-c77f82949579 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.916079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a21580-ab04-4041-82f0-c77f82949579 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.917420 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 806/1618] 10.222.0.79 () {54 vars in 912 bytes} [Tue Sep 6 21:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:13:41.919747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a21580-ab04-4041-82f0-c77f82949579 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:41.937432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a21580-ab04-4041-82f0-c77f82949579 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:41.942295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e959c-7dcd-4c7e-9090-7ba4d20ae4e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:41.943173 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e959c-7dcd-4c7e-9090-7ba4d20ae4e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:41.943765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e959c-7dcd-4c7e-9090-7ba4d20ae4e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:41.944047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e959c-7dcd-4c7e-9090-7ba4d20ae4e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:41.966027 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:42.032175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e959c-7dcd-4c7e-9090-7ba4d20ae4e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:42.035128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e959c-7dcd-4c7e-9090-7ba4d20ae4e9 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:42.037991 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a16160-7bf2-4d15-a5bb-285d83726586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:42.040324 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a16160-7bf2-4d15-a5bb-285d83726586 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:42.056949 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 807/1619] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:42.061576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e959c-7dcd-4c7e-9090-7ba4d20ae4e9 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:42.069694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 813/1620] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:42.073710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a16160-7bf2-4d15-a5bb-285d83726586 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:42.079028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e959c-7dcd-4c7e-9090-7ba4d20ae4e9 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:42.085176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655db2c-e97d-491e-ab37-830cc684ee6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:42.085788 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655db2c-e97d-491e-ab37-830cc684ee6f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:42.086072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655db2c-e97d-491e-ab37-830cc684ee6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:42.086345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655db2c-e97d-491e-ab37-830cc684ee6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:42.100546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a16160-7bf2-4d15-a5bb-285d83726586 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:42.103137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655db2c-e97d-491e-ab37-830cc684ee6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:42.104826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655db2c-e97d-491e-ab37-830cc684ee6f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:42.116478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324a21f-13b2-48e7-904d-eaac8b41f8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:42.116478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324a21f-13b2-48e7-904d-eaac8b41f8ec admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:42.116822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324a21f-13b2-48e7-904d-eaac8b41f8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:42.118846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324a21f-13b2-48e7-904d-eaac8b41f8ec admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:42.119584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324a21f-13b2-48e7-904d-eaac8b41f8ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:42.123482 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324a21f-13b2-48e7-904d-eaac8b41f8ec admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:42.139074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 808/1621] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:42.140558 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf95826-9057-4f86-95b9-ad0873e901b6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:42.141978 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf95826-9057-4f86-95b9-ad0873e901b6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:42.142487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf95826-9057-4f86-95b9-ad0873e901b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:42.169863 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-acf95826-9057-4f86-95b9-ad0873e901b6 None None] MFA Rules not processed for user `12d51454d20c46d7a5e6d963a2f987cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:42.171273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-acf95826-9057-4f86-95b9-ad0873e901b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:42.192803 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 814/1622] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:13:42 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:42.194683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7e0fc-4f40-4ca7-b602-d07ba4b9228b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:42.195068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7e0fc-4f40-4ca7-b602-d07ba4b9228b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:42.195554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7e0fc-4f40-4ca7-b602-d07ba4b9228b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:42.197424 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 815/1623] 10.222.0.79 () {54 vars in 962 bytes} [Tue Sep 6 21:13:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:13:42.216485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7e0fc-4f40-4ca7-b602-d07ba4b9228b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:42.234030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7e0fc-4f40-4ca7-b602-d07ba4b9228b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:42.241914 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32353de6-dfeb-4d72-8f29-05cd7be3042d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:42.243048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32353de6-dfeb-4d72-8f29-05cd7be3042d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:42.243174 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32353de6-dfeb-4d72-8f29-05cd7be3042d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:42.243467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32353de6-dfeb-4d72-8f29-05cd7be3042d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:42.244396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32353de6-dfeb-4d72-8f29-05cd7be3042d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:42.247232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32353de6-dfeb-4d72-8f29-05cd7be3042d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:42.267663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 816/1624] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:42.267663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456f641-cf1b-4174-9afd-cde4aea8f9f3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:42.267663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456f641-cf1b-4174-9afd-cde4aea8f9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:42.267663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456f641-cf1b-4174-9afd-cde4aea8f9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:42.290212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5456f641-cf1b-4174-9afd-cde4aea8f9f3 None None] MFA Rules not processed for user `12d51454d20c46d7a5e6d963a2f987cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:42.291494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5456f641-cf1b-4174-9afd-cde4aea8f9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:42.314021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:42.340989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 809/1625] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:42.341978 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf95826-9057-4f86-95b9-ad0873e901b6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:42.363811 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf95826-9057-4f86-95b9-ad0873e901b6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:42.370218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac184d63-5c45-48fc-9ab6-2c7710670f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:42.370916 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac184d63-5c45-48fc-9ab6-2c7710670f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:42.371195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac184d63-5c45-48fc-9ab6-2c7710670f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:42.371542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac184d63-5c45-48fc-9ab6-2c7710670f8f admin admin] PATH_INFO: `/v3/projects/8ff70bd9b22a45dc9fe098a9f6615cfe/users/2914b8408d4d47d1bc5f3547ad32ed5b/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:42.409766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac184d63-5c45-48fc-9ab6-2c7710670f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff70bd9b22a45dc9fe098a9f6615cfe, user_id=2914b8408d4d47d1bc5f3547ad32ed5b, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:42.471670 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac184d63-5c45-48fc-9ab6-2c7710670f8f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:42.512777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 817/1626] 10.222.0.79 () {58 vars in 1059 bytes} [Tue Sep 6 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:42.512777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456f641-cf1b-4174-9afd-cde4aea8f9f3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:42.513985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 810/1627] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:42 2022] PUT /identity/v3/projects/8ff70bd9b22a45dc9fe098a9f6615cfe/users/2914b8408d4d47d1bc5f3547ad32ed5b/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:42.513985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac184d63-5c45-48fc-9ab6-2c7710670f8f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:42.566094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac184d63-5c45-48fc-9ab6-2c7710670f8f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:42.571156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456f641-cf1b-4174-9afd-cde4aea8f9f3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:42.573437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08607941-54bb-4a62-af5b-6365c74bc9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:42.574340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08607941-54bb-4a62-af5b-6365c74bc9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:42.574815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08607941-54bb-4a62-af5b-6365c74bc9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:42.575421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08607941-54bb-4a62-af5b-6365c74bc9d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:42.576398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08607941-54bb-4a62-af5b-6365c74bc9d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:42.579012 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08607941-54bb-4a62-af5b-6365c74bc9d6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:42.620037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 811/1628] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:42.622265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08607941-54bb-4a62-af5b-6365c74bc9d6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:42.640582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:42.670480 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef164af-f70c-4069-988e-f978b2a46414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:42.671474 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef164af-f70c-4069-988e-f978b2a46414 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:42.671984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef164af-f70c-4069-988e-f978b2a46414 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:42.672434 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef164af-f70c-4069-988e-f978b2a46414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:42.772135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08607941-54bb-4a62-af5b-6365c74bc9d6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:42.782017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef164af-f70c-4069-988e-f978b2a46414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:42.783378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de158f99-73ba-4082-9edb-d3a98c9ad529 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:42.783974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef164af-f70c-4069-988e-f978b2a46414 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:42.784305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158f99-73ba-4082-9edb-d3a98c9ad529 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:42.784725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158f99-73ba-4082-9edb-d3a98c9ad529 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:42.785174 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158f99-73ba-4082-9edb-d3a98c9ad529 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:42.798248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 818/1629] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:42.799204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef164af-f70c-4069-988e-f978b2a46414 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:42.813159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef164af-f70c-4069-988e-f978b2a46414 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:42.820232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f62dea-27b5-4406-b2af-36943df09a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:42.821046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f62dea-27b5-4406-b2af-36943df09a69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:42.821655 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f62dea-27b5-4406-b2af-36943df09a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:42.822020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f62dea-27b5-4406-b2af-36943df09a69 admin admin] PATH_INFO: `/v3/projects/8ff70bd9b22a45dc9fe098a9f6615cfe/users/2914b8408d4d47d1bc5f3547ad32ed5b/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:42.825086 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f62dea-27b5-4406-b2af-36943df09a69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff70bd9b22a45dc9fe098a9f6615cfe, user_id=2914b8408d4d47d1bc5f3547ad32ed5b, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:42.829413 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f62dea-27b5-4406-b2af-36943df09a69 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:42.847502 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 819/1630] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:42 2022] PUT /identity/v3/projects/8ff70bd9b22a45dc9fe098a9f6615cfe/users/2914b8408d4d47d1bc5f3547ad32ed5b/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:42.858785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ab225-7479-4d52-a064-024d41070c9a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:42.859020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ab225-7479-4d52-a064-024d41070c9a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:42.859291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ab225-7479-4d52-a064-024d41070c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:42.942695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fe8ab225-7479-4d52-a064-024d41070c9a None None] MFA Rules not processed for user `2914b8408d4d47d1bc5f3547ad32ed5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:42.943934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fe8ab225-7479-4d52-a064-024d41070c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:42.951262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de158f99-73ba-4082-9edb-d3a98c9ad529 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:42.954811 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de158f99-73ba-4082-9edb-d3a98c9ad529 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:42.963858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:42.978910 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 812/1631] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:43.020581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 820/1632] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:13:43.033456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de158f99-73ba-4082-9edb-d3a98c9ad529 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:43.050796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de158f99-73ba-4082-9edb-d3a98c9ad529 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:43.128026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de37df-f782-4b57-a4c0-48b2e922b214 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:43.128791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de37df-f782-4b57-a4c0-48b2e922b214 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:43.129079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de37df-f782-4b57-a4c0-48b2e922b214 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:43.129480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de37df-f782-4b57-a4c0-48b2e922b214 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:43.141982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de37df-f782-4b57-a4c0-48b2e922b214 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:43.143786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de37df-f782-4b57-a4c0-48b2e922b214 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:43.180200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 813/1633] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:43.357780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ab225-7479-4d52-a064-024d41070c9a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:43.375382 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ab225-7479-4d52-a064-024d41070c9a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:43.387778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9742926-f7cb-49e7-baee-6b240bee2114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:43.391538 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9742926-f7cb-49e7-baee-6b240bee2114 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:43.393569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9742926-f7cb-49e7-baee-6b240bee2114 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:43.393890 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9742926-f7cb-49e7-baee-6b240bee2114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:43.472442 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a071a4-b7ca-4261-bf08-b26f2464e08c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:43.472860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a071a4-b7ca-4261-bf08-b26f2464e08c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:43.473079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a071a4-b7ca-4261-bf08-b26f2464e08c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:43.474990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 814/1634] 10.222.0.79 () {54 vars in 912 bytes} [Tue Sep 6 21:13:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:13:43.482462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6cd60-9c7f-4662-8d35-6b2e3da47e96 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:43.482858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6cd60-9c7f-4662-8d35-6b2e3da47e96 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:43.483138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6cd60-9c7f-4662-8d35-6b2e3da47e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:43.530897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-78f6cd60-9c7f-4662-8d35-6b2e3da47e96 None None] MFA Rules not processed for user `12d51454d20c46d7a5e6d963a2f987cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:43.537974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-78f6cd60-9c7f-4662-8d35-6b2e3da47e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:43.553838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:43.585094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9742926-f7cb-49e7-baee-6b240bee2114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:43.586790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9742926-f7cb-49e7-baee-6b240bee2114 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:43.621157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 821/1635] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:43.769378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 815/1636] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:13:43.910173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9742926-f7cb-49e7-baee-6b240bee2114 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:43.929760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9742926-f7cb-49e7-baee-6b240bee2114 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:43.944454 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d81beb-ce94-486e-b238-46c4cfbe0859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:43.945564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d81beb-ce94-486e-b238-46c4cfbe0859 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:43.945998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d81beb-ce94-486e-b238-46c4cfbe0859 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:43.947016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d81beb-ce94-486e-b238-46c4cfbe0859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:43.987841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d81beb-ce94-486e-b238-46c4cfbe0859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:43.993636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d81beb-ce94-486e-b238-46c4cfbe0859 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:44.033767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 822/1637] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:46.074081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec994a2-49ee-4214-bfd6-5e6827ec7978 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:46.074950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec994a2-49ee-4214-bfd6-5e6827ec7978 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:46.075352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec994a2-49ee-4214-bfd6-5e6827ec7978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:46.149889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5ec994a2-49ee-4214-bfd6-5e6827ec7978 None None] MFA Rules not processed for user `ae3562403ce64a25b206d9a9fdbb6673`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:46.151364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5ec994a2-49ee-4214-bfd6-5e6827ec7978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:46.305378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 816/1638] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:46.309979 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d81beb-ce94-486e-b238-46c4cfbe0859 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:46.449212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d81beb-ce94-486e-b238-46c4cfbe0859 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:46.462829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99eac58-f572-4343-a04d-be1eb70c5438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:46.463768 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99eac58-f572-4343-a04d-be1eb70c5438 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:46.464874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99eac58-f572-4343-a04d-be1eb70c5438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:46.465310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99eac58-f572-4343-a04d-be1eb70c5438 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:46.468722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99eac58-f572-4343-a04d-be1eb70c5438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:46.471836 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99eac58-f572-4343-a04d-be1eb70c5438 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:46.538853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 823/1639] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:13:46 2022] POST /identity/v3/projects => generated 405 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:46.542953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec994a2-49ee-4214-bfd6-5e6827ec7978 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:46.559212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec994a2-49ee-4214-bfd6-5e6827ec7978 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:46.610191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:46.673427 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9594e3-c8da-48b2-ae10-764785bac133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:46.674223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9594e3-c8da-48b2-ae10-764785bac133 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:46.674640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9594e3-c8da-48b2-ae10-764785bac133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:46.675390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9594e3-c8da-48b2-ae10-764785bac133 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:46.676141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9594e3-c8da-48b2-ae10-764785bac133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:46.678217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9594e3-c8da-48b2-ae10-764785bac133 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:46.706048 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 817/1640] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:13:46 2022] POST /identity/v3/users => generated 309 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:46.710253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99eac58-f572-4343-a04d-be1eb70c5438 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:46.722930 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99eac58-f572-4343-a04d-be1eb70c5438 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:46.728101 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b5ede-890e-4f73-8c27-dd3d9eb6616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:46.728758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b5ede-890e-4f73-8c27-dd3d9eb6616e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:46.729172 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b5ede-890e-4f73-8c27-dd3d9eb6616e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:46.729648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b5ede-890e-4f73-8c27-dd3d9eb6616e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:46.730397 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b5ede-890e-4f73-8c27-dd3d9eb6616e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:46.732178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b5ede-890e-4f73-8c27-dd3d9eb6616e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:46.744642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 824/1641] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:46.749042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9594e3-c8da-48b2-ae10-764785bac133 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:46.764278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9594e3-c8da-48b2-ae10-764785bac133 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:46.770480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00004350-f93c-4749-93bf-638d619dd992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:46.771334 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00004350-f93c-4749-93bf-638d619dd992 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:46.771784 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00004350-f93c-4749-93bf-638d619dd992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:46.772325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00004350-f93c-4749-93bf-638d619dd992 admin admin] PATH_INFO: `/v3/projects/a3673e5ad8fe45b69fba7816c39aa0c9/users/cc701b790d7f42bf8f8ea152330b9a9f/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:46.797926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00004350-f93c-4749-93bf-638d619dd992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3673e5ad8fe45b69fba7816c39aa0c9, user_id=cc701b790d7f42bf8f8ea152330b9a9f, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:46.801840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00004350-f93c-4749-93bf-638d619dd992 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:46.828520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 818/1642] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:46 2022] PUT /identity/v3/projects/a3673e5ad8fe45b69fba7816c39aa0c9/users/cc701b790d7f42bf8f8ea152330b9a9f/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:46.832837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b5ede-890e-4f73-8c27-dd3d9eb6616e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:46.844861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b5ede-890e-4f73-8c27-dd3d9eb6616e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:46.910489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26a9f7-4746-44e5-adb0-d4db5551ae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:46.911214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26a9f7-4746-44e5-adb0-d4db5551ae62 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:46.911606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26a9f7-4746-44e5-adb0-d4db5551ae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:46.911984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26a9f7-4746-44e5-adb0-d4db5551ae62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:46.912783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26a9f7-4746-44e5-adb0-d4db5551ae62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:46.914782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26a9f7-4746-44e5-adb0-d4db5551ae62 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:46.926827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 825/1643] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:46.931314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00004350-f93c-4749-93bf-638d619dd992 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:46.943973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00004350-f93c-4749-93bf-638d619dd992 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:46.949867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b53925-f705-4cae-9d7f-6033d4a60043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:46.951692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b53925-f705-4cae-9d7f-6033d4a60043 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:46.952011 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b53925-f705-4cae-9d7f-6033d4a60043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:46.952387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b53925-f705-4cae-9d7f-6033d4a60043 admin admin] PATH_INFO: `/v3/projects/a3673e5ad8fe45b69fba7816c39aa0c9/users/cc701b790d7f42bf8f8ea152330b9a9f/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:46.954624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b53925-f705-4cae-9d7f-6033d4a60043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3673e5ad8fe45b69fba7816c39aa0c9, user_id=cc701b790d7f42bf8f8ea152330b9a9f, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:46.957946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b53925-f705-4cae-9d7f-6033d4a60043 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:46.981044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 819/1644] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:46 2022] PUT /identity/v3/projects/a3673e5ad8fe45b69fba7816c39aa0c9/users/cc701b790d7f42bf8f8ea152330b9a9f/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:48.821907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72b5d9-e31d-4ea8-9b3b-eb6f93469c7a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:48.822838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72b5d9-e31d-4ea8-9b3b-eb6f93469c7a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:48.823706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72b5d9-e31d-4ea8-9b3b-eb6f93469c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:48.964678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fe72b5d9-e31d-4ea8-9b3b-eb6f93469c7a None None] MFA Rules not processed for user `cc701b790d7f42bf8f8ea152330b9a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:48.977635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fe72b5d9-e31d-4ea8-9b3b-eb6f93469c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:49.066445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:49.144266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 826/1645] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:48 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:51.911798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186d45c-642c-41ca-94c9-11162c178fd8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:51.912317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186d45c-642c-41ca-94c9-11162c178fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:51.912317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186d45c-642c-41ca-94c9-11162c178fd8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:51.914313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 820/1646] 10.222.0.79 () {54 vars in 921 bytes} [Tue Sep 6 21:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:13:51.948917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91670cf8-e1a7-48af-88db-24802f9c1f4e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:51.949834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91670cf8-e1a7-48af-88db-24802f9c1f4e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:51.950373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91670cf8-e1a7-48af-88db-24802f9c1f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:51.988761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-91670cf8-e1a7-48af-88db-24802f9c1f4e None None] MFA Rules not processed for user `c40b5ff21d1940749bf479cc936fc0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:51.995251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-91670cf8-e1a7-48af-88db-24802f9c1f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:52.167540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 827/1647] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Sep 6 21:13:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:52.198092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186d45c-642c-41ca-94c9-11162c178fd8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:52.218052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186d45c-642c-41ca-94c9-11162c178fd8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:52.232926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4571a9-0581-47c8-a9ad-2f936e7a1c55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:52.233987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4571a9-0581-47c8-a9ad-2f936e7a1c55 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:52.234998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4571a9-0581-47c8-a9ad-2f936e7a1c55 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:52.236128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4571a9-0581-47c8-a9ad-2f936e7a1c55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:52.252330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4571a9-0581-47c8-a9ad-2f936e7a1c55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:52.254512 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4571a9-0581-47c8-a9ad-2f936e7a1c55 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:52.280796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 821/1648] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:53.150817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc50765-7193-44bd-806f-b33f10e6fe81 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:53.151431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc50765-7193-44bd-806f-b33f10e6fe81 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:53.151525 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc50765-7193-44bd-806f-b33f10e6fe81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:53.240074 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cdc50765-7193-44bd-806f-b33f10e6fe81 None None] MFA Rules not processed for user `54fe3d99f05c45f099804fccb4679503`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:53.241350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cdc50765-7193-44bd-806f-b33f10e6fe81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:53.439734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 828/1649] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:53.446527 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4571a9-0581-47c8-a9ad-2f936e7a1c55 service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:53.688749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4571a9-0581-47c8-a9ad-2f936e7a1c55 service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:53.713481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d51e98-6fff-4e3d-95a6-d31eea7ff2b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:53.714950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d51e98-6fff-4e3d-95a6-d31eea7ff2b6 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:53.719401 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d51e98-6fff-4e3d-95a6-d31eea7ff2b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:53.721979 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d51e98-6fff-4e3d-95a6-d31eea7ff2b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:53.761840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d51e98-6fff-4e3d-95a6-d31eea7ff2b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:53.770237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d51e98-6fff-4e3d-95a6-d31eea7ff2b6 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:53.805549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 822/1650] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:54.638844 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc50765-7193-44bd-806f-b33f10e6fe81 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:54.714263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:54.724978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc50765-7193-44bd-806f-b33f10e6fe81 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:54.737765 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:54.738598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:54.738942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:54.739199 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] PATH_INFO: `/v3/users/8f011304deec4c798bff42fbb8cf4c09` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:54.741080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f011304deec4c798bff42fbb8cf4c09)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:54.745288 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:54.836881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f011304deec4c798bff42fbb8cf4c09'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:54.837733 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f011304deec4c798bff42fbb8cf4c09'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:54.849201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f011304deec4c798bff42fbb8cf4c09'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:54.875785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f011304deec4c798bff42fbb8cf4c09'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:54.877549 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f011304deec4c798bff42fbb8cf4c09'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:54.918272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 829/1651] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:13:54 2022] DELETE /identity/v3/users/8f011304deec4c798bff42fbb8cf4c09 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:54.922626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe20cb3-0192-4eab-96a8-77e651ecd49d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:54.923564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe20cb3-0192-4eab-96a8-77e651ecd49d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:54.924041 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe20cb3-0192-4eab-96a8-77e651ecd49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:54.965854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fbe20cb3-0192-4eab-96a8-77e651ecd49d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:54.967237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fbe20cb3-0192-4eab-96a8-77e651ecd49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:55.100179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:55.156431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 823/1652] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:55.228576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:55.230393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:55.373500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a4770-bb4b-40b9-b831-fbfd81be4202 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:55.379920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf126e4-7e04-4dc1-97b6-aa75a674399b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:55.381032 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf126e4-7e04-4dc1-97b6-aa75a674399b service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:55.381440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf126e4-7e04-4dc1-97b6-aa75a674399b service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:55.381936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf126e4-7e04-4dc1-97b6-aa75a674399b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:55.410356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf126e4-7e04-4dc1-97b6-aa75a674399b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:55.417087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf126e4-7e04-4dc1-97b6-aa75a674399b service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:55.452330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 830/1653] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:55.481963 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ac97c-c3be-4482-b5bf-82c2e6f3d1e8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:55.482343 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ac97c-c3be-4482-b5bf-82c2e6f3d1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:55.482670 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ac97c-c3be-4482-b5bf-82c2e6f3d1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:55.528091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2d7ac97c-c3be-4482-b5bf-82c2e6f3d1e8 None None] MFA Rules not processed for user `64033668b9fd4ba48eef09bdc3fdf2ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:55.529628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2d7ac97c-c3be-4482-b5bf-82c2e6f3d1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:55.755737 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 824/1654] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:55.773494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf126e4-7e04-4dc1-97b6-aa75a674399b service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:55.777014 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bbf126e4-7e04-4dc1-97b6-aa75a674399b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:55.881687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104cf78-b99c-46fd-b499-155e449b309c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:55.881687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104cf78-b99c-46fd-b499-155e449b309c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:55.881687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104cf78-b99c-46fd-b499-155e449b309c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:55.883648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 825/1655] 10.222.0.79 () {54 vars in 921 bytes} [Tue Sep 6 21:13:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:13:55.922236 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead504e-51b3-49ab-b8c3-f4b776aa7024 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:55.922703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead504e-51b3-49ab-b8c3-f4b776aa7024 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:55.923275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead504e-51b3-49ab-b8c3-f4b776aa7024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:55.966168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf126e4-7e04-4dc1-97b6-aa75a674399b service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:55.973640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4850b-a630-45aa-9512-10d9b3c259fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:55.974533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0ead504e-51b3-49ab-b8c3-f4b776aa7024 None None] MFA Rules not processed for user `c40b5ff21d1940749bf479cc936fc0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:55.975767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4850b-a630-45aa-9512-10d9b3c259fc service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:55.976788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4850b-a630-45aa-9512-10d9b3c259fc service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:55.977218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4850b-a630-45aa-9512-10d9b3c259fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:55.977707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0ead504e-51b3-49ab-b8c3-f4b776aa7024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:55.995917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4850b-a630-45aa-9512-10d9b3c259fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:55.998482 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4850b-a630-45aa-9512-10d9b3c259fc service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:56.012395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 831/1656] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:56.081380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4850b-a630-45aa-9512-10d9b3c259fc service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:56.082954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-48b4850b-a630-45aa-9512-10d9b3c259fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:56.098802 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 826/1657] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Sep 6 21:13:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:56.099217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:56.111761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4850b-a630-45aa-9512-10d9b3c259fc service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:56.119968 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673841eb-9a2b-4ffc-a0ad-71ba394a89b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:56.121097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673841eb-9a2b-4ffc-a0ad-71ba394a89b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:56.123429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673841eb-9a2b-4ffc-a0ad-71ba394a89b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:56.128828 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673841eb-9a2b-4ffc-a0ad-71ba394a89b2 admin admin] PATH_INFO: `/v3/users/ae3562403ce64a25b206d9a9fdbb6673` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:56.134523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673841eb-9a2b-4ffc-a0ad-71ba394a89b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae3562403ce64a25b206d9a9fdbb6673)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:56.136794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673841eb-9a2b-4ffc-a0ad-71ba394a89b2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:56.137858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead504e-51b3-49ab-b8c3-f4b776aa7024 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:56.141638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0ead504e-51b3-49ab-b8c3-f4b776aa7024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:56.172721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead504e-51b3-49ab-b8c3-f4b776aa7024 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:56.187399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a470bd-23a5-44b5-b798-f95a327d3551 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:56.188415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a470bd-23a5-44b5-b798-f95a327d3551 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:56.188930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a470bd-23a5-44b5-b798-f95a327d3551 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:56.189535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a470bd-23a5-44b5-b798-f95a327d3551 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:56.204247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a470bd-23a5-44b5-b798-f95a327d3551 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:56.206559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a470bd-23a5-44b5-b798-f95a327d3551 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:56.226164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 827/1658] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:56.272107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-673841eb-9a2b-4ffc-a0ad-71ba394a89b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae3562403ce64a25b206d9a9fdbb6673'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:56.273216 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-673841eb-9a2b-4ffc-a0ad-71ba394a89b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae3562403ce64a25b206d9a9fdbb6673'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:56.290724 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-673841eb-9a2b-4ffc-a0ad-71ba394a89b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae3562403ce64a25b206d9a9fdbb6673'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:56.323533 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-673841eb-9a2b-4ffc-a0ad-71ba394a89b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae3562403ce64a25b206d9a9fdbb6673'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:56.324166 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-673841eb-9a2b-4ffc-a0ad-71ba394a89b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae3562403ce64a25b206d9a9fdbb6673'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:56.362046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 832/1659] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:13:56 2022] DELETE /identity/v3/users/ae3562403ce64a25b206d9a9fdbb6673 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:13:56.431274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002751ad-c36b-4676-b52e-8186fe189361 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:56.431829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002751ad-c36b-4676-b52e-8186fe189361 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:56.432209 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002751ad-c36b-4676-b52e-8186fe189361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:56.523094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-002751ad-c36b-4676-b52e-8186fe189361 None None] MFA Rules not processed for user `2ec47d6e2e9b40a6bd207c084041f7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:56.529280 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-002751ad-c36b-4676-b52e-8186fe189361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:56.663487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:56.695308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 828/1660] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:56.705838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6160-d5c1-455e-bb9e-df532e36c367 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:56.706594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6160-d5c1-455e-bb9e-df532e36c367 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:56.710130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6160-d5c1-455e-bb9e-df532e36c367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:56.772038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-472c6160-d5c1-455e-bb9e-df532e36c367 None None] MFA Rules not processed for user `2ec47d6e2e9b40a6bd207c084041f7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:56.773777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-472c6160-d5c1-455e-bb9e-df532e36c367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:56.807680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 833/1661] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:56.820581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002751ad-c36b-4676-b52e-8186fe189361 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:56.822312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-002751ad-c36b-4676-b52e-8186fe189361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:56.842118 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c6160-d5c1-455e-bb9e-df532e36c367 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:56.844513 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-472c6160-d5c1-455e-bb9e-df532e36c367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:56.988172 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c6160-d5c1-455e-bb9e-df532e36c367 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:56.994598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:56.995522 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:56.995939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:56.996347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e admin admin] PATH_INFO: `/v3/projects/8e2d0b17f13b4989bbb230a60d6c5662` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:56.997741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e2d0b17f13b4989bbb230a60d6c5662)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:56.999016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:13:56.999016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 21:13:56.999016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: Sep 06 21:13:57.000797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:57.054495 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002751ad-c36b-4676-b52e-8186fe189361 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:57.072681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48670a76-dc70-4dc7-8fa0-43d81ca9a2f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:57.073717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48670a76-dc70-4dc7-8fa0-43d81ca9a2f4 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:57.074251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48670a76-dc70-4dc7-8fa0-43d81ca9a2f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:57.074733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48670a76-dc70-4dc7-8fa0-43d81ca9a2f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:57.093042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48670a76-dc70-4dc7-8fa0-43d81ca9a2f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:57.095849 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48670a76-dc70-4dc7-8fa0-43d81ca9a2f4 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:57.104483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e2d0b17f13b4989bbb230a60d6c5662', 'request_id': 'req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:57.120345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e2d0b17f13b4989bbb230a60d6c5662', 'request_id': 'req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:57.127891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 829/1662] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:56 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:57.142012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e2d0b17f13b4989bbb230a60d6c5662', 'request_id': 'req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:57.178995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:57.183702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e admin admin] The token cache is being invalidate because project 8e2d0b17f13b4989bbb230a60d6c5662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:13:57.184204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:57.185892 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3e5c9a7-d9f2-4733-9524-c84436dbc82e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:57.191783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 834/1663] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:13:56 2022] DELETE /identity/v3/projects/8e2d0b17f13b4989bbb230a60d6c5662 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:57.193689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ef36d-f1a0-478d-9218-b71171bd21ec None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:57.194040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ef36d-f1a0-478d-9218-b71171bd21ec None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:57.194376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ef36d-f1a0-478d-9218-b71171bd21ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:57.237068 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8e8ef36d-f1a0-478d-9218-b71171bd21ec None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:57.241505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8e8ef36d-f1a0-478d-9218-b71171bd21ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:57.376218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 830/1664] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:57.387156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ae2da-bbe6-4132-87fa-fefefd3aba0e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:57.387156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ae2da-bbe6-4132-87fa-fefefd3aba0e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:57.387156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ae2da-bbe6-4132-87fa-fefefd3aba0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:57.415365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fa7ae2da-bbe6-4132-87fa-fefefd3aba0e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:57.416630 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fa7ae2da-bbe6-4132-87fa-fefefd3aba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:57.431564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 835/1665] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:57.432258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ef36d-f1a0-478d-9218-b71171bd21ec None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:57.446067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ef36d-f1a0-478d-9218-b71171bd21ec None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:57.457211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420e2e3-0138-493c-88a4-913a097e73da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:57.457853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420e2e3-0138-493c-88a4-913a097e73da admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:57.458287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420e2e3-0138-493c-88a4-913a097e73da admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:57.458689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420e2e3-0138-493c-88a4-913a097e73da admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:57.459510 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420e2e3-0138-493c-88a4-913a097e73da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:13:57.459957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420e2e3-0138-493c-88a4-913a097e73da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:57.463494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420e2e3-0138-493c-88a4-913a097e73da admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:57.493903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 831/1666] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:13:57.498912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ae2da-bbe6-4132-87fa-fefefd3aba0e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:57.513930 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ae2da-bbe6-4132-87fa-fefefd3aba0e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:57.527478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f597a-6dcc-4dc0-9b7a-c086c9c934cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:57.528583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f597a-6dcc-4dc0-9b7a-c086c9c934cd admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:57.530238 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f597a-6dcc-4dc0-9b7a-c086c9c934cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:57.530571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f597a-6dcc-4dc0-9b7a-c086c9c934cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:57.531556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f597a-6dcc-4dc0-9b7a-c086c9c934cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:57.535932 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f597a-6dcc-4dc0-9b7a-c086c9c934cd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:57.617864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 836/1667] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:13:57 2022] POST /identity/v3/projects => generated 363 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:57.618905 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420e2e3-0138-493c-88a4-913a097e73da admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:57.637658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420e2e3-0138-493c-88a4-913a097e73da admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:57.806878 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602f9f8-bd04-4970-b11b-1e9b8da5cb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:57.807874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602f9f8-bd04-4970-b11b-1e9b8da5cb3e admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:57.808513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602f9f8-bd04-4970-b11b-1e9b8da5cb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:57.808973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602f9f8-bd04-4970-b11b-1e9b8da5cb3e admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:57.810173 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602f9f8-bd04-4970-b11b-1e9b8da5cb3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:57.823824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602f9f8-bd04-4970-b11b-1e9b8da5cb3e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:57.853092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:57.890006 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 832/1668] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:13:57 2022] POST /identity/v3/users => generated 282 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:57.894641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f597a-6dcc-4dc0-9b7a-c086c9c934cd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:57.910446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f597a-6dcc-4dc0-9b7a-c086c9c934cd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:57.925452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7baac6c-39b3-4313-9cf2-e7928c344ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:57.926249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7baac6c-39b3-4313-9cf2-e7928c344ead admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:57.926638 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7baac6c-39b3-4313-9cf2-e7928c344ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:57.927043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7baac6c-39b3-4313-9cf2-e7928c344ead admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:57.927831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7baac6c-39b3-4313-9cf2-e7928c344ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:57.931814 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7baac6c-39b3-4313-9cf2-e7928c344ead admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:57.975452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602f9f8-bd04-4970-b11b-1e9b8da5cb3e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:57.976212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 837/1669] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:57.990875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602f9f8-bd04-4970-b11b-1e9b8da5cb3e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:58.008191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64a7e1-8324-452a-b53e-e3ce0ff3b81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:58.008191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64a7e1-8324-452a-b53e-e3ce0ff3b81a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:58.008191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64a7e1-8324-452a-b53e-e3ce0ff3b81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:58.010564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64a7e1-8324-452a-b53e-e3ce0ff3b81a admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:58.011251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64a7e1-8324-452a-b53e-e3ce0ff3b81a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:58.013008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64a7e1-8324-452a-b53e-e3ce0ff3b81a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:58.047998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 833/1670] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:58.049680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7baac6c-39b3-4313-9cf2-e7928c344ead admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:58.071654 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64a7e1-8324-452a-b53e-e3ce0ff3b81a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:58.073609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dc64a7e1-8324-452a-b53e-e3ce0ff3b81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:58.075673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7baac6c-39b3-4313-9cf2-e7928c344ead admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:58.087302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02e5c8-c289-49a9-8290-ed1aeb5f7fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:58.088168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02e5c8-c289-49a9-8290-ed1aeb5f7fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:58.088583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02e5c8-c289-49a9-8290-ed1aeb5f7fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:58.089034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02e5c8-c289-49a9-8290-ed1aeb5f7fd5 admin admin] PATH_INFO: `/v3/projects/bb4de354a72248aaad0ed4c4470d31f1/users/5e3921eefd564142975e382317f47b4a/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:58.103560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64a7e1-8324-452a-b53e-e3ce0ff3b81a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:58.117207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:58.117876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:58.118157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:58.118431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] PATH_INFO: `/v3/projects/341f27c705cf4f05b7de4a6c001eb661` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:58.122075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=341f27c705cf4f05b7de4a6c001eb661)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:58.124028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:58.151309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02e5c8-c289-49a9-8290-ed1aeb5f7fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb4de354a72248aaad0ed4c4470d31f1, user_id=5e3921eefd564142975e382317f47b4a, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:58.157955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02e5c8-c289-49a9-8290-ed1aeb5f7fd5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:58.184248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 838/1671] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:58 2022] PUT /identity/v3/projects/bb4de354a72248aaad0ed4c4470d31f1/users/5e3921eefd564142975e382317f47b4a/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:58.198004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfce68-ad72-4421-9241-6aeebab376f2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:58.198364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfce68-ad72-4421-9241-6aeebab376f2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:58.199412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfce68-ad72-4421-9241-6aeebab376f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:58.236036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '341f27c705cf4f05b7de4a6c001eb661', 'request_id': 'req-eb0bb417-18df-4e0d-832a-306d93e3b679'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:58.243503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2cbfce68-ad72-4421-9241-6aeebab376f2 None None] MFA Rules not processed for user `5e3921eefd564142975e382317f47b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:58.243503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2cbfce68-ad72-4421-9241-6aeebab376f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:58.269073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '341f27c705cf4f05b7de4a6c001eb661', 'request_id': 'req-eb0bb417-18df-4e0d-832a-306d93e3b679'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:58.279725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '341f27c705cf4f05b7de4a6c001eb661', 'request_id': 'req-eb0bb417-18df-4e0d-832a-306d93e3b679'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:58.304920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] The token cache is being invalidate because project 341f27c705cf4f05b7de4a6c001eb661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:13:58.305211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:58.305896 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:58.307683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 834/1672] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:13:58 2022] DELETE /identity/v3/projects/341f27c705cf4f05b7de4a6c001eb661 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:58.315852 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:58.317353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:58.369612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 839/1673] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:58.370303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfce68-ad72-4421-9241-6aeebab376f2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:58.373581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2cbfce68-ad72-4421-9241-6aeebab376f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:58.468325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfce68-ad72-4421-9241-6aeebab376f2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:58.476166 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de581a19-e121-4b93-8b8f-2f97abff989b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:58.476755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bb417-18df-4e0d-832a-306d93e3b679 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:58.476880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de581a19-e121-4b93-8b8f-2f97abff989b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:58.477124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de581a19-e121-4b93-8b8f-2f97abff989b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:58.477442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de581a19-e121-4b93-8b8f-2f97abff989b admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:58.478268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de581a19-e121-4b93-8b8f-2f97abff989b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:58.483099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de581a19-e121-4b93-8b8f-2f97abff989b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:58.495377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65120ba3-8c98-44bc-911a-356044367aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:58.495939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65120ba3-8c98-44bc-911a-356044367aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:58.496220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65120ba3-8c98-44bc-911a-356044367aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:58.496488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65120ba3-8c98-44bc-911a-356044367aeb admin admin] PATH_INFO: `/v3/users/cc701b790d7f42bf8f8ea152330b9a9f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:58.497954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65120ba3-8c98-44bc-911a-356044367aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc701b790d7f42bf8f8ea152330b9a9f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:58.499176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-65120ba3-8c98-44bc-911a-356044367aeb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 21:13:58.499176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 21:13:58.499176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: Sep 06 21:13:58.500598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65120ba3-8c98-44bc-911a-356044367aeb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:58.544326 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 840/1674] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:13:58 2022] POST /identity/v3/projects => generated 361 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:58.545843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c8496-3543-489d-b92c-eb878d9dabc1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:58.546138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c8496-3543-489d-b92c-eb878d9dabc1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:58.546425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c8496-3543-489d-b92c-eb878d9dabc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:58.625292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:58.651954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2a9c8496-3543-489d-b92c-eb878d9dabc1 None None] MFA Rules not processed for user `c914e6a41ccc48f49194d57551bdf0d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:58.661776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2a9c8496-3543-489d-b92c-eb878d9dabc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:58.712187 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-65120ba3-8c98-44bc-911a-356044367aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc701b790d7f42bf8f8ea152330b9a9f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:58.713761 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-65120ba3-8c98-44bc-911a-356044367aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc701b790d7f42bf8f8ea152330b9a9f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:58.730369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-65120ba3-8c98-44bc-911a-356044367aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc701b790d7f42bf8f8ea152330b9a9f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:58.770824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=2,keystone:SELECT=83,keystone:DELETE=15 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:58.788375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-65120ba3-8c98-44bc-911a-356044367aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc701b790d7f42bf8f8ea152330b9a9f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:58.790824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-65120ba3-8c98-44bc-911a-356044367aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc701b790d7f42bf8f8ea152330b9a9f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:13:58.824307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 841/1675] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:58.829904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 835/1676] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:13:58 2022] DELETE /identity/v3/users/cc701b790d7f42bf8f8ea152330b9a9f => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:58.835158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c8496-3543-489d-b92c-eb878d9dabc1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:58.835442 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb1a75-02d5-4946-aa54-8469ea25eaea None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:58.835737 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb1a75-02d5-4946-aa54-8469ea25eaea None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:58.836015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb1a75-02d5-4946-aa54-8469ea25eaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:58.842488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2a9c8496-3543-489d-b92c-eb878d9dabc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:58.886598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-61eb1a75-02d5-4946-aa54-8469ea25eaea None None] MFA Rules not processed for user `c914e6a41ccc48f49194d57551bdf0d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:58.890570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-61eb1a75-02d5-4946-aa54-8469ea25eaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:58.998354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c8496-3543-489d-b92c-eb878d9dabc1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:59.049363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 836/1677] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:59.052191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c2f35-e795-4082-aa73-232aaac29aa0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:59.052482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c2f35-e795-4082-aa73-232aaac29aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:59.052783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c2f35-e795-4082-aa73-232aaac29aa0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:59.074579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 837/1678] 10.222.0.79 () {54 vars in 912 bytes} [Tue Sep 6 21:13:59 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:13:59.077793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c2f35-e795-4082-aa73-232aaac29aa0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:59.084426 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-866c2f35-e795-4082-aa73-232aaac29aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:59.126773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370d501-ff32-48eb-89b6-54570d874080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:59.127583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370d501-ff32-48eb-89b6-54570d874080 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:59.128026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370d501-ff32-48eb-89b6-54570d874080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:59.128394 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370d501-ff32-48eb-89b6-54570d874080 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:59.129775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370d501-ff32-48eb-89b6-54570d874080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:59.132839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370d501-ff32-48eb-89b6-54570d874080 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:59.187623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 842/1679] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:13:58 2022] POST /identity/v3/users => generated 287 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:13:59.192189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15df54-ddbb-473d-89e9-fc4f4e1c85ef None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:59.192771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15df54-ddbb-473d-89e9-fc4f4e1c85ef None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:59.193417 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15df54-ddbb-473d-89e9-fc4f4e1c85ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:59.259557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fb15df54-ddbb-473d-89e9-fc4f4e1c85ef None None] MFA Rules not processed for user `d5a45dec40844877a6b5b301fc44a110`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:59.261184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fb15df54-ddbb-473d-89e9-fc4f4e1c85ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:59.271462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c2f35-e795-4082-aa73-232aaac29aa0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:59.281054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfb33b-f06c-4bd5-a974-00dd9b53b959 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:59.281578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfb33b-f06c-4bd5-a974-00dd9b53b959 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:59.281862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfb33b-f06c-4bd5-a974-00dd9b53b959 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:59.282118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfb33b-f06c-4bd5-a974-00dd9b53b959 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:59.284965 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:59.309009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfb33b-f06c-4bd5-a974-00dd9b53b959 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:59.311014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfb33b-f06c-4bd5-a974-00dd9b53b959 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:59.332672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 838/1680] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:59.333644 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfb33b-f06c-4bd5-a974-00dd9b53b959 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:59.355081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfb33b-f06c-4bd5-a974-00dd9b53b959 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:59.358362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 843/1681] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:13:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:13:59.370360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664336e-0090-4e82-86b0-6a4a80571bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:59.371013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664336e-0090-4e82-86b0-6a4a80571bed admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:59.371294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664336e-0090-4e82-86b0-6a4a80571bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:59.371563 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664336e-0090-4e82-86b0-6a4a80571bed admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:59.372285 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664336e-0090-4e82-86b0-6a4a80571bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:59.373989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664336e-0090-4e82-86b0-6a4a80571bed admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:59.396735 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 839/1682] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:59.399149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15df54-ddbb-473d-89e9-fc4f4e1c85ef None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:59.424809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15df54-ddbb-473d-89e9-fc4f4e1c85ef None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:59.445208 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0c867-9b6b-426e-b9bd-9dfecee50d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:59.447306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0c867-9b6b-426e-b9bd-9dfecee50d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:59.448442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0c867-9b6b-426e-b9bd-9dfecee50d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:59.449536 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0c867-9b6b-426e-b9bd-9dfecee50d82 admin admin] PATH_INFO: `/v3/projects/f520442d6d0d4c66b80a52c352193ee7/users/4ff123eac06941db9def6fc39c285785/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:59.512352 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0c867-9b6b-426e-b9bd-9dfecee50d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f520442d6d0d4c66b80a52c352193ee7, user_id=4ff123eac06941db9def6fc39c285785, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:59.525808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0c867-9b6b-426e-b9bd-9dfecee50d82 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:59.540975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664336e-0090-4e82-86b0-6a4a80571bed admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:59.542665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2664336e-0090-4e82-86b0-6a4a80571bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:13:59.561985 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 844/1683] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:59 2022] PUT /identity/v3/projects/f520442d6d0d4c66b80a52c352193ee7/users/4ff123eac06941db9def6fc39c285785/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:13:59.564380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0c867-9b6b-426e-b9bd-9dfecee50d82 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:59.567138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664336e-0090-4e82-86b0-6a4a80571bed admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:59.707327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f9fa7-29ef-4ea5-9340-e22e9d2289fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:59.708001 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f9fa7-29ef-4ea5-9340-e22e9d2289fd service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:59.708282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f9fa7-29ef-4ea5-9340-e22e9d2289fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:59.708555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f9fa7-29ef-4ea5-9340-e22e9d2289fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:59.711422 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0c867-9b6b-426e-b9bd-9dfecee50d82 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:59.722969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca58152-f2a1-4edc-84c7-4d1c930f3cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:59.723597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca58152-f2a1-4edc-84c7-4d1c930f3cfe admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:59.723869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca58152-f2a1-4edc-84c7-4d1c930f3cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:59.724133 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca58152-f2a1-4edc-84c7-4d1c930f3cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:59.724814 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca58152-f2a1-4edc-84c7-4d1c930f3cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:59.726436 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca58152-f2a1-4edc-84c7-4d1c930f3cfe admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:59.732293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f9fa7-29ef-4ea5-9340-e22e9d2289fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:59.734278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f9fa7-29ef-4ea5-9340-e22e9d2289fd service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:59.752960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 845/1684] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:13:59.754231 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 840/1685] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:13:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:13:59.756869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca58152-f2a1-4edc-84c7-4d1c930f3cfe admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:13:59.785507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca58152-f2a1-4edc-84c7-4d1c930f3cfe admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:13:59.804923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3ead1-2f27-4b3f-aa50-b9543aaf92b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:13:59.808892 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3ead1-2f27-4b3f-aa50-b9543aaf92b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:59.809657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3ead1-2f27-4b3f-aa50-b9543aaf92b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:59.810227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3ead1-2f27-4b3f-aa50-b9543aaf92b0 admin admin] PATH_INFO: `/v3/projects/f520442d6d0d4c66b80a52c352193ee7/users/4ff123eac06941db9def6fc39c285785/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:59.815988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3ead1-2f27-4b3f-aa50-b9543aaf92b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f520442d6d0d4c66b80a52c352193ee7, user_id=4ff123eac06941db9def6fc39c285785, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:13:59.824103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3ead1-2f27-4b3f-aa50-b9543aaf92b0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:13:59.867386 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 846/1686] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:13:59 2022] PUT /identity/v3/projects/f520442d6d0d4c66b80a52c352193ee7/users/4ff123eac06941db9def6fc39c285785/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:13:59.881048 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cd4ce-c91c-4e0b-a1ab-307d6f6af305 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:13:59.881378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cd4ce-c91c-4e0b-a1ab-307d6f6af305 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:13:59.881651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cd4ce-c91c-4e0b-a1ab-307d6f6af305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:13:59.908822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:13:59.945375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0f5cd4ce-c91c-4e0b-a1ab-307d6f6af305 None None] MFA Rules not processed for user `4ff123eac06941db9def6fc39c285785`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:13:59.951003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0f5cd4ce-c91c-4e0b-a1ab-307d6f6af305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:00.126708 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 841/1687] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:13:59 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:00.139797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3ead1-2f27-4b3f-aa50-b9543aaf92b0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:00.142363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0bc3ead1-2f27-4b3f-aa50-b9543aaf92b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:00.250354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cd4ce-c91c-4e0b-a1ab-307d6f6af305 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:00.251905 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0f5cd4ce-c91c-4e0b-a1ab-307d6f6af305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:00.253562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3ead1-2f27-4b3f-aa50-b9543aaf92b0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:00.262483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abedd71-a11b-4819-ad00-154330902a09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:00.263160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abedd71-a11b-4819-ad00-154330902a09 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:00.263566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abedd71-a11b-4819-ad00-154330902a09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:00.263842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abedd71-a11b-4819-ad00-154330902a09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:00.295194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abedd71-a11b-4819-ad00-154330902a09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:00.296875 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abedd71-a11b-4819-ad00-154330902a09 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:00.330026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 847/1688] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:00 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:00.374555 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa248d6-3665-44d3-a775-79a1a3d8e7cc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:00.374860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa248d6-3665-44d3-a775-79a1a3d8e7cc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:00.375161 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa248d6-3665-44d3-a775-79a1a3d8e7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:00.421537 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2aa248d6-3665-44d3-a775-79a1a3d8e7cc None None] MFA Rules not processed for user `54fe3d99f05c45f099804fccb4679503`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:00.424550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2aa248d6-3665-44d3-a775-79a1a3d8e7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:00.539802 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cd4ce-c91c-4e0b-a1ab-307d6f6af305 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:00.550552 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:00.551151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:00.551407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:00.552073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] PATH_INFO: `/v3/projects/a3673e5ad8fe45b69fba7816c39aa0c9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:00.559846 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3673e5ad8fe45b69fba7816c39aa0c9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:00.559846 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:00.561559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:00.607290 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 848/1689] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:14:00.608817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce776d9-dd64-4e42-b413-1dc2f7f2dee6 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:00.609111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce776d9-dd64-4e42-b413-1dc2f7f2dee6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:00.609353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce776d9-dd64-4e42-b413-1dc2f7f2dee6 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:00.610658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 849/1690] 10.222.0.79 () {54 vars in 912 bytes} [Tue Sep 6 21:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:14:00.626731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a900c-616b-4b72-80f1-0c40041acab9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:00.627046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a900c-616b-4b72-80f1-0c40041acab9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:00.627326 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a900c-616b-4b72-80f1-0c40041acab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:00.665016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-437a900c-616b-4b72-80f1-0c40041acab9 None None] MFA Rules not processed for user `d5a45dec40844877a6b5b301fc44a110`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:00.666265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-437a900c-616b-4b72-80f1-0c40041acab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:00.725189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3673e5ad8fe45b69fba7816c39aa0c9', 'request_id': 'req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:00.757397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3673e5ad8fe45b69fba7816c39aa0c9', 'request_id': 'req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:00.768235 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3673e5ad8fe45b69fba7816c39aa0c9', 'request_id': 'req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:00.780554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 850/1691] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:00.794737 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] The token cache is being invalidate because project a3673e5ad8fe45b69fba7816c39aa0c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:14:00.795039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:00.795877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:00.798003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 842/1692] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:14:00 2022] DELETE /identity/v3/projects/a3673e5ad8fe45b69fba7816c39aa0c9 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:00.803256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23929478-1ca5-4997-bc39-3eb24c8f9f21 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:00.803544 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23929478-1ca5-4997-bc39-3eb24c8f9f21 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:00.803825 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23929478-1ca5-4997-bc39-3eb24c8f9f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:00.820336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:00.831389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:00.876599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-23929478-1ca5-4997-bc39-3eb24c8f9f21 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:00.880801 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-23929478-1ca5-4997-bc39-3eb24c8f9f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:00.903569 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:01.035632 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51b3b5-5872-41d1-947d-6cbd2a7e5d70 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:01.043366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d64f0-df04-4ce1-b2b7-086496fb41f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:01.044949 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d64f0-df04-4ce1-b2b7-086496fb41f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:01.045773 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d64f0-df04-4ce1-b2b7-086496fb41f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:01.046975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d64f0-df04-4ce1-b2b7-086496fb41f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:01.061677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-427d64f0-df04-4ce1-b2b7-086496fb41f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:01.088392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d64f0-df04-4ce1-b2b7-086496fb41f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:01.099853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d64f0-df04-4ce1-b2b7-086496fb41f8 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:01.124088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 843/1693] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:01.125244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d64f0-df04-4ce1-b2b7-086496fb41f8 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:01.126810 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-427d64f0-df04-4ce1-b2b7-086496fb41f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:01.159128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 851/1694] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:01.160985 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dc3bb-668a-49fb-a6f1-9599acf64bfc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:01.161460 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dc3bb-668a-49fb-a6f1-9599acf64bfc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:01.161578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dc3bb-668a-49fb-a6f1-9599acf64bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:01.247134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-387dc3bb-668a-49fb-a6f1-9599acf64bfc None None] MFA Rules not processed for user `2ec47d6e2e9b40a6bd207c084041f7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:01.256414 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-387dc3bb-668a-49fb-a6f1-9599acf64bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:01.418786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d64f0-df04-4ce1-b2b7-086496fb41f8 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:01.429307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b1313-94ce-4f13-a991-a59c150dc2bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:01.430076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b1313-94ce-4f13-a991-a59c150dc2bc service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:01.430361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b1313-94ce-4f13-a991-a59c150dc2bc service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:01.430630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b1313-94ce-4f13-a991-a59c150dc2bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:01.447743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-705b1313-94ce-4f13-a991-a59c150dc2bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:01.454129 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 852/1695] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:01.455994 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37e6a5-8377-4504-a68f-2f3222f36b69 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:01.458168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37e6a5-8377-4504-a68f-2f3222f36b69 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:01.458699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37e6a5-8377-4504-a68f-2f3222f36b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:01.497098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b1313-94ce-4f13-a991-a59c150dc2bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:01.498701 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b1313-94ce-4f13-a991-a59c150dc2bc service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:01.506221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6a37e6a5-8377-4504-a68f-2f3222f36b69 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:01.510143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6a37e6a5-8377-4504-a68f-2f3222f36b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:01.531871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 844/1696] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:01.532670 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b1313-94ce-4f13-a991-a59c150dc2bc service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:01.537563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 853/1697] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:01.538842 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-705b1313-94ce-4f13-a991-a59c150dc2bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:01.539038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37e6a5-8377-4504-a68f-2f3222f36b69 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:01.557776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37e6a5-8377-4504-a68f-2f3222f36b69 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:01.580330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63f8c7-7782-4489-a8cb-2944dc1b59ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:01.581453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63f8c7-7782-4489-a8cb-2944dc1b59ff admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:01.581956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63f8c7-7782-4489-a8cb-2944dc1b59ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:01.582259 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b1313-94ce-4f13-a991-a59c150dc2bc service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:01.583136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63f8c7-7782-4489-a8cb-2944dc1b59ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:01.585610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63f8c7-7782-4489-a8cb-2944dc1b59ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:14:01.587928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63f8c7-7782-4489-a8cb-2944dc1b59ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:01.597241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63f8c7-7782-4489-a8cb-2944dc1b59ff admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:01.600141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9607fc5-ddb7-42d0-813b-1cc818328e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:01.600866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9607fc5-ddb7-42d0-813b-1cc818328e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:01.601119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9607fc5-ddb7-42d0-813b-1cc818328e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:01.609627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9607fc5-ddb7-42d0-813b-1cc818328e25 admin admin] PATH_INFO: `/v3/users/5e3921eefd564142975e382317f47b4a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:01.615052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9607fc5-ddb7-42d0-813b-1cc818328e25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e3921eefd564142975e382317f47b4a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:01.617063 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9607fc5-ddb7-42d0-813b-1cc818328e25 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:01.628009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 854/1698] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:14:01.633136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63f8c7-7782-4489-a8cb-2944dc1b59ff admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:01.683505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63f8c7-7782-4489-a8cb-2944dc1b59ff admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:01.702566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6758d4-3aac-417a-bf67-4167000b670c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:01.703404 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6758d4-3aac-417a-bf67-4167000b670c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:01.703799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6758d4-3aac-417a-bf67-4167000b670c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:01.719110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6758d4-3aac-417a-bf67-4167000b670c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:01.723697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6758d4-3aac-417a-bf67-4167000b670c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:01.734245 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6758d4-3aac-417a-bf67-4167000b670c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:01.743018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c9607fc5-ddb7-42d0-813b-1cc818328e25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e3921eefd564142975e382317f47b4a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:01.743876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c9607fc5-ddb7-42d0-813b-1cc818328e25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e3921eefd564142975e382317f47b4a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:01.758721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c9607fc5-ddb7-42d0-813b-1cc818328e25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e3921eefd564142975e382317f47b4a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:01.766449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:01.786176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c9607fc5-ddb7-42d0-813b-1cc818328e25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e3921eefd564142975e382317f47b4a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:01.786988 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c9607fc5-ddb7-42d0-813b-1cc818328e25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e3921eefd564142975e382317f47b4a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:01.795967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 855/1699] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:14:01 2022] POST /identity/v3/projects => generated 399 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:01.800350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6758d4-3aac-417a-bf67-4167000b670c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:01.804543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ac6758d4-3aac-417a-bf67-4167000b670c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:01.817035 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 845/1700] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:14:01 2022] DELETE /identity/v3/users/5e3921eefd564142975e382317f47b4a => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:01.822202 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574f8a7-e42c-401d-b63e-68ebe9148cd7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:01.822510 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574f8a7-e42c-401d-b63e-68ebe9148cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:01.822782 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574f8a7-e42c-401d-b63e-68ebe9148cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:01.872722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3574f8a7-e42c-401d-b63e-68ebe9148cd7 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:01.878463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3574f8a7-e42c-401d-b63e-68ebe9148cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:01.954775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:02.019269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6758d4-3aac-417a-bf67-4167000b670c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:02.074312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 846/1701] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:14:02.074312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d83bbb-2c43-4718-b7a2-a05aed17f99e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:02.074312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d83bbb-2c43-4718-b7a2-a05aed17f99e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:02.074312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d83bbb-2c43-4718-b7a2-a05aed17f99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:02.108462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-19d83bbb-2c43-4718-b7a2-a05aed17f99e None None] MFA Rules not processed for user `64033668b9fd4ba48eef09bdc3fdf2ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:02.110242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-19d83bbb-2c43-4718-b7a2-a05aed17f99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:02.180273 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef88410-50f0-4dd1-bace-ae940f4d2b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:02.180938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef88410-50f0-4dd1-bace-ae940f4d2b66 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:02.181219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef88410-50f0-4dd1-bace-ae940f4d2b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:02.181492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef88410-50f0-4dd1-bace-ae940f4d2b66 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:02.182145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef88410-50f0-4dd1-bace-ae940f4d2b66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:02.184201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef88410-50f0-4dd1-bace-ae940f4d2b66 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:02.256068 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 847/1702] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:02.257409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d83bbb-2c43-4718-b7a2-a05aed17f99e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:02.260999 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-19d83bbb-2c43-4718-b7a2-a05aed17f99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:02.279590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 856/1703] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:14:01 2022] POST /identity/v3/users => generated 300 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:02.280925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef88410-50f0-4dd1-bace-ae940f4d2b66 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:02.356599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef88410-50f0-4dd1-bace-ae940f4d2b66 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:02.436535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9506a0-6ba2-4d67-80d2-efcff103f229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:02.437552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9506a0-6ba2-4d67-80d2-efcff103f229 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:02.438974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9506a0-6ba2-4d67-80d2-efcff103f229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:02.441639 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9506a0-6ba2-4d67-80d2-efcff103f229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:02.445744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9506a0-6ba2-4d67-80d2-efcff103f229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:02.463119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9506a0-6ba2-4d67-80d2-efcff103f229 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:02.506006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 857/1704] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:02.510952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9506a0-6ba2-4d67-80d2-efcff103f229 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:02.536020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9506a0-6ba2-4d67-80d2-efcff103f229 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:02.554184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d83bbb-2c43-4718-b7a2-a05aed17f99e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:02.557185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89990f-417b-4b4e-a680-c187ea128802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:02.558090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89990f-417b-4b4e-a680-c187ea128802 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:02.558632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89990f-417b-4b4e-a680-c187ea128802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:02.559233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89990f-417b-4b4e-a680-c187ea128802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:02.560027 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89990f-417b-4b4e-a680-c187ea128802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:02.561915 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89990f-417b-4b4e-a680-c187ea128802 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:02.565348 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f021c-1c86-4aab-8ab3-5298f8fa7364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:02.568947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f021c-1c86-4aab-8ab3-5298f8fa7364 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:02.569247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f021c-1c86-4aab-8ab3-5298f8fa7364 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:02.569529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f021c-1c86-4aab-8ab3-5298f8fa7364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:02.608022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 858/1705] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:02.614370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89990f-417b-4b4e-a680-c187ea128802 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:02.624603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f021c-1c86-4aab-8ab3-5298f8fa7364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:02.631616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f021c-1c86-4aab-8ab3-5298f8fa7364 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:02.640295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89990f-417b-4b4e-a680-c187ea128802 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:02.656065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60a83f-cfbc-4f02-bfc4-0deff537ea08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:02.656996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60a83f-cfbc-4f02-bfc4-0deff537ea08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:02.657433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60a83f-cfbc-4f02-bfc4-0deff537ea08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:02.657839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60a83f-cfbc-4f02-bfc4-0deff537ea08 admin admin] PATH_INFO: `/v3/projects/4c5895bb13774751b365bfcfea2ba68d/users/f63913fa87f240ce8c39190d51a4b9de/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:02.666767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 848/1706] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:02.760348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60a83f-cfbc-4f02-bfc4-0deff537ea08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5895bb13774751b365bfcfea2ba68d, user_id=f63913fa87f240ce8c39190d51a4b9de, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:02.763307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60a83f-cfbc-4f02-bfc4-0deff537ea08 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:02.797464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 859/1707] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:02 2022] PUT /identity/v3/projects/4c5895bb13774751b365bfcfea2ba68d/users/f63913fa87f240ce8c39190d51a4b9de/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:02.801998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3fea2-3c4c-4e27-afd0-d60c0bcca4a3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:02.802460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3fea2-3c4c-4e27-afd0-d60c0bcca4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:02.806449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3fea2-3c4c-4e27-afd0-d60c0bcca4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:02.929717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3dd3fea2-3c4c-4e27-afd0-d60c0bcca4a3 None None] MFA Rules not processed for user `f63913fa87f240ce8c39190d51a4b9de`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:02.931133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3dd3fea2-3c4c-4e27-afd0-d60c0bcca4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:03.105798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 849/1708] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:03.111676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60a83f-cfbc-4f02-bfc4-0deff537ea08 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:03.295099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:03.332714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60a83f-cfbc-4f02-bfc4-0deff537ea08 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:03.344085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092390d-1463-43d1-b4a0-40d03e0b10d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:03.345614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092390d-1463-43d1-b4a0-40d03e0b10d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:03.347327 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092390d-1463-43d1-b4a0-40d03e0b10d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:03.348756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092390d-1463-43d1-b4a0-40d03e0b10d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:03.350664 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092390d-1463-43d1-b4a0-40d03e0b10d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:03.356868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092390d-1463-43d1-b4a0-40d03e0b10d2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:03.473060 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 860/1709] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:14:03 2022] POST /identity/v3/projects => generated 399 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:03.477230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3fea2-3c4c-4e27-afd0-d60c0bcca4a3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:03.485629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:03.497121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3fea2-3c4c-4e27-afd0-d60c0bcca4a3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:03.568978 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eede69-eb43-43fe-996b-15bbb2db6efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:03.569840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eede69-eb43-43fe-996b-15bbb2db6efa admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:03.570301 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eede69-eb43-43fe-996b-15bbb2db6efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:03.570759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eede69-eb43-43fe-996b-15bbb2db6efa admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:03.571798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eede69-eb43-43fe-996b-15bbb2db6efa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:03.574459 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eede69-eb43-43fe-996b-15bbb2db6efa admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:03.639895 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 850/1710] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:14:03 2022] POST /identity/v3/users => generated 306 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:03.644578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092390d-1463-43d1-b4a0-40d03e0b10d2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:03.668907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092390d-1463-43d1-b4a0-40d03e0b10d2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:03.692752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a82d4b-3165-466d-991f-c7828fc84932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:03.693814 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a82d4b-3165-466d-991f-c7828fc84932 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:03.694250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a82d4b-3165-466d-991f-c7828fc84932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:03.694673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a82d4b-3165-466d-991f-c7828fc84932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:03.703662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a82d4b-3165-466d-991f-c7828fc84932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:03.705382 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a82d4b-3165-466d-991f-c7828fc84932 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:03.752146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 861/1711] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:03.756500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eede69-eb43-43fe-996b-15bbb2db6efa admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:03.779845 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eede69-eb43-43fe-996b-15bbb2db6efa admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:03.797966 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf1e80-1571-40be-8290-7c79c9e8dee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:03.799148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf1e80-1571-40be-8290-7c79c9e8dee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:03.799462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf1e80-1571-40be-8290-7c79c9e8dee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:03.799738 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf1e80-1571-40be-8290-7c79c9e8dee7 admin admin] PATH_INFO: `/v3/projects/d7b17da3dd5f49169e3c9f6640ea384a/users/2bc9f2a7ace943a09185a2f7f83bb80b/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:03.850560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf1e80-1571-40be-8290-7c79c9e8dee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7b17da3dd5f49169e3c9f6640ea384a, user_id=2bc9f2a7ace943a09185a2f7f83bb80b, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:03.860142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf1e80-1571-40be-8290-7c79c9e8dee7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:03.898512 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 851/1712] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:03 2022] PUT /identity/v3/projects/d7b17da3dd5f49169e3c9f6640ea384a/users/2bc9f2a7ace943a09185a2f7f83bb80b/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:03.902450 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a82d4b-3165-466d-991f-c7828fc84932 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:03.936782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a82d4b-3165-466d-991f-c7828fc84932 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:03.966623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf1e80-1571-40be-8290-7c79c9e8dee7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:03.968214 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4eaf1e80-1571-40be-8290-7c79c9e8dee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:04.312598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfabc7a-2d0e-4621-8f93-c32583ad3f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:04.318306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfabc7a-2d0e-4621-8f93-c32583ad3f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:04.320034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfabc7a-2d0e-4621-8f93-c32583ad3f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:04.320322 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfabc7a-2d0e-4621-8f93-c32583ad3f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:04.323674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfabc7a-2d0e-4621-8f93-c32583ad3f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:04.325486 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfabc7a-2d0e-4621-8f93-c32583ad3f6c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:04.355510 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf1e80-1571-40be-8290-7c79c9e8dee7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:04.381176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:04.386232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 862/1713] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 484 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:04.387114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:04.387656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:04.388131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] PATH_INFO: `/v3/projects/bb4de354a72248aaad0ed4c4470d31f1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:04.390832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfabc7a-2d0e-4621-8f93-c32583ad3f6c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:04.393315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb4de354a72248aaad0ed4c4470d31f1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:04.405997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:04.436234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfabc7a-2d0e-4621-8f93-c32583ad3f6c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:04.456337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b7ce3-27ae-4f76-a685-6378cc4e26d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:04.459259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b7ce3-27ae-4f76-a685-6378cc4e26d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:04.459864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b7ce3-27ae-4f76-a685-6378cc4e26d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:04.462028 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b7ce3-27ae-4f76-a685-6378cc4e26d3 admin admin] PATH_INFO: `/v3/projects/d7b17da3dd5f49169e3c9f6640ea384a/users/2bc9f2a7ace943a09185a2f7f83bb80b/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:04.475185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b7ce3-27ae-4f76-a685-6378cc4e26d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7b17da3dd5f49169e3c9f6640ea384a, user_id=2bc9f2a7ace943a09185a2f7f83bb80b, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:04.478359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b7ce3-27ae-4f76-a685-6378cc4e26d3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:04.542562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 863/1714] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:04 2022] PUT /identity/v3/projects/d7b17da3dd5f49169e3c9f6640ea384a/users/2bc9f2a7ace943a09185a2f7f83bb80b/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:04.555268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a5d69-e46a-4823-984f-b7a0bc108dfc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:04.560620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a5d69-e46a-4823-984f-b7a0bc108dfc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:04.566611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a5d69-e46a-4823-984f-b7a0bc108dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:04.645684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb4de354a72248aaad0ed4c4470d31f1', 'request_id': 'req-4e95a593-bc52-40e4-ab9b-801f27ad6a87'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:04.684983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb4de354a72248aaad0ed4c4470d31f1', 'request_id': 'req-4e95a593-bc52-40e4-ab9b-801f27ad6a87'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:04.714172 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb4de354a72248aaad0ed4c4470d31f1', 'request_id': 'req-4e95a593-bc52-40e4-ab9b-801f27ad6a87'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:04.717770 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-502a5d69-e46a-4823-984f-b7a0bc108dfc None None] MFA Rules not processed for user `2bc9f2a7ace943a09185a2f7f83bb80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:04.720190 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-502a5d69-e46a-4823-984f-b7a0bc108dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:04.741091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:04.761775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] The token cache is being invalidate because project bb4de354a72248aaad0ed4c4470d31f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:14:04.765131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:04.769306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:04.774913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 852/1715] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:14:03 2022] DELETE /identity/v3/projects/bb4de354a72248aaad0ed4c4470d31f1 => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:04.781213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:04.787241 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:04.933838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 864/1716] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:04 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:04.935695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a5d69-e46a-4823-984f-b7a0bc108dfc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:04.937241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-502a5d69-e46a-4823-984f-b7a0bc108dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:05.007457 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:05.094811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a5d69-e46a-4823-984f-b7a0bc108dfc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:05.100244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95a593-bc52-40e4-ab9b-801f27ad6a87 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:05.108908 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:05.110977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:05.111683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:05.112410 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] PATH_INFO: `/v3/users/8bf3e32edd3a41daaecfe6c73a761363` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:05.113992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a1409-64f4-4263-abf3-9fce383899f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:05.114841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf3e32edd3a41daaecfe6c73a761363)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:05.115013 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a1409-64f4-4263-abf3-9fce383899f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:05.115322 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a1409-64f4-4263-abf3-9fce383899f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:05.115642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a1409-64f4-4263-abf3-9fce383899f3 admin admin] PATH_INFO: `/v3/users/4ff123eac06941db9def6fc39c285785` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:05.119213 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a1409-64f4-4263-abf3-9fce383899f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ff123eac06941db9def6fc39c285785)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:05.121382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:05.126339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a1409-64f4-4263-abf3-9fce383899f3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:05.257806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf3e32edd3a41daaecfe6c73a761363'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:05.261453 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf3e32edd3a41daaecfe6c73a761363'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:05.274695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-578a1409-64f4-4263-abf3-9fce383899f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ff123eac06941db9def6fc39c285785'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:05.276358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-578a1409-64f4-4263-abf3-9fce383899f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ff123eac06941db9def6fc39c285785'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:05.315397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf3e32edd3a41daaecfe6c73a761363'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:05.343774 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-578a1409-64f4-4263-abf3-9fce383899f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ff123eac06941db9def6fc39c285785'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:05.398497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf3e32edd3a41daaecfe6c73a761363'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:05.402617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf3e32edd3a41daaecfe6c73a761363'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:05.413582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-578a1409-64f4-4263-abf3-9fce383899f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ff123eac06941db9def6fc39c285785'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:05.416394 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-578a1409-64f4-4263-abf3-9fce383899f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ff123eac06941db9def6fc39c285785'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:05.483151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 853/1717] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:14:04 2022] DELETE /identity/v3/users/8bf3e32edd3a41daaecfe6c73a761363 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:05.490286 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:05.500797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:05.501811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 865/1718] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:14:04 2022] DELETE /identity/v3/users/4ff123eac06941db9def6fc39c285785 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:05.509916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470c438-96bb-47f4-a443-e21fae1c5031 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:05.511752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470c438-96bb-47f4-a443-e21fae1c5031 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:05.512233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470c438-96bb-47f4-a443-e21fae1c5031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:05.647829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e470c438-96bb-47f4-a443-e21fae1c5031 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:05.658741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e470c438-96bb-47f4-a443-e21fae1c5031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:05.773829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a28e9d-26a7-42cb-a57c-daab8c13b63a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:05.875433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 866/1719] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:05.922669 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:05.961537 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470c438-96bb-47f4-a443-e21fae1c5031 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:05.963137 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e470c438-96bb-47f4-a443-e21fae1c5031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:05.968520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5fd64-9096-4b61-8abd-5a7a4a4fd6d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:05.969673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5fd64-9096-4b61-8abd-5a7a4a4fd6d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:05.970057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5fd64-9096-4b61-8abd-5a7a4a4fd6d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:05.970403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5fd64-9096-4b61-8abd-5a7a4a4fd6d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:05.973115 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-01b5fd64-9096-4b61-8abd-5a7a4a4fd6d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:06.122670 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:06.137924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470c438-96bb-47f4-a443-e21fae1c5031 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:06.141848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5fd64-9096-4b61-8abd-5a7a4a4fd6d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:06.143944 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5fd64-9096-4b61-8abd-5a7a4a4fd6d7 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:06.148622 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10500fc-11c3-489d-a3a0-f138368b7d3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:06.149280 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10500fc-11c3-489d-a3a0-f138368b7d3a service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:06.149550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10500fc-11c3-489d-a3a0-f138368b7d3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:06.149819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10500fc-11c3-489d-a3a0-f138368b7d3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:06.158790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 854/1720] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:05 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:06.216609 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10500fc-11c3-489d-a3a0-f138368b7d3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:06.222494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10500fc-11c3-489d-a3a0-f138368b7d3a service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:06.238387 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 867/1721] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:06.406876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5fd64-9096-4b61-8abd-5a7a4a4fd6d7 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:06.409419 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-01b5fd64-9096-4b61-8abd-5a7a4a4fd6d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:06.436275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5fd64-9096-4b61-8abd-5a7a4a4fd6d7 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:06.449617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13a7ba-a1bc-4520-9722-b365ead75b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:06.452956 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13a7ba-a1bc-4520-9722-b365ead75b33 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:06.455128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13a7ba-a1bc-4520-9722-b365ead75b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:06.455565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13a7ba-a1bc-4520-9722-b365ead75b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:06.468617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6b13a7ba-a1bc-4520-9722-b365ead75b33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:06.690035 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13a7ba-a1bc-4520-9722-b365ead75b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:06.691540 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13a7ba-a1bc-4520-9722-b365ead75b33 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:06.722084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 855/1722] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:06 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:07.009467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10500fc-11c3-489d-a3a0-f138368b7d3a service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:07.011962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a10500fc-11c3-489d-a3a0-f138368b7d3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:07.036757 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10500fc-11c3-489d-a3a0-f138368b7d3a service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:07.045876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403a418-0dad-4da4-843e-d35d65415a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:07.047703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403a418-0dad-4da4-843e-d35d65415a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:07.047980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403a418-0dad-4da4-843e-d35d65415a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:07.048697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403a418-0dad-4da4-843e-d35d65415a99 admin admin] PATH_INFO: `/v3/projects/f520442d6d0d4c66b80a52c352193ee7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:07.051504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403a418-0dad-4da4-843e-d35d65415a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f520442d6d0d4c66b80a52c352193ee7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:07.058792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403a418-0dad-4da4-843e-d35d65415a99 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:07.116575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76698c-bef5-474a-b198-d4dc177c5ff6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:07.117221 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76698c-bef5-474a-b198-d4dc177c5ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:07.117221 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76698c-bef5-474a-b198-d4dc177c5ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:07.208568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3403a418-0dad-4da4-843e-d35d65415a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f520442d6d0d4c66b80a52c352193ee7', 'request_id': 'req-3403a418-0dad-4da4-843e-d35d65415a99'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:07.241111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3403a418-0dad-4da4-843e-d35d65415a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f520442d6d0d4c66b80a52c352193ee7', 'request_id': 'req-3403a418-0dad-4da4-843e-d35d65415a99'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:07.255671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3403a418-0dad-4da4-843e-d35d65415a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f520442d6d0d4c66b80a52c352193ee7', 'request_id': 'req-3403a418-0dad-4da4-843e-d35d65415a99'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:07.288148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0c76698c-bef5-474a-b198-d4dc177c5ff6 None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:07.291900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0c76698c-bef5-474a-b198-d4dc177c5ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:07.303406 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3403a418-0dad-4da4-843e-d35d65415a99 admin admin] The token cache is being invalidate because project f520442d6d0d4c66b80a52c352193ee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:14:07.303722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3403a418-0dad-4da4-843e-d35d65415a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:07.307393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3403a418-0dad-4da4-843e-d35d65415a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:07.316971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 868/1723] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:14:07 2022] DELETE /identity/v3/projects/f520442d6d0d4c66b80a52c352193ee7 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:14:07.342370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22906a7-d007-47bc-ab9a-06d8b579e2cd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:07.342807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22906a7-d007-47bc-ab9a-06d8b579e2cd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:07.347251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22906a7-d007-47bc-ab9a-06d8b579e2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:07.458983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a22906a7-d007-47bc-ab9a-06d8b579e2cd None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:07.473127 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a22906a7-d007-47bc-ab9a-06d8b579e2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:07.511213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:07.560750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 856/1724] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 445 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 06 21:14:07.648106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 869/1725] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:07.656558 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7e861-d0ae-4cd4-bb9b-92ea9bca896b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:07.657078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7e861-d0ae-4cd4-bb9b-92ea9bca896b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:07.657550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7e861-d0ae-4cd4-bb9b-92ea9bca896b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:07.714557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-72f7e861-d0ae-4cd4-bb9b-92ea9bca896b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:07.716576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-72f7e861-d0ae-4cd4-bb9b-92ea9bca896b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:07.727769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 857/1726] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:14:07.732146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22906a7-d007-47bc-ab9a-06d8b579e2cd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:07.747787 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22906a7-d007-47bc-ab9a-06d8b579e2cd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:07.756774 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ce219-585b-42a6-8edc-c48ce28166a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:07.757924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ce219-585b-42a6-8edc-c48ce28166a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:07.758794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ce219-585b-42a6-8edc-c48ce28166a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:07.759532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ce219-585b-42a6-8edc-c48ce28166a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:07.760884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ce219-585b-42a6-8edc-c48ce28166a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:14:07.761605 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ce219-585b-42a6-8edc-c48ce28166a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:07.764092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ce219-585b-42a6-8edc-c48ce28166a6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:07.781163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 870/1727] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:14:07.785888 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7e861-d0ae-4cd4-bb9b-92ea9bca896b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:07.800523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ce219-585b-42a6-8edc-c48ce28166a6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:07.802108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7e861-d0ae-4cd4-bb9b-92ea9bca896b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:07.802515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3e5ce219-585b-42a6-8edc-c48ce28166a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:07.807554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3ed8b-8969-46b7-b939-f38a0c297465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:07.809082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3ed8b-8969-46b7-b939-f38a0c297465 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:07.809435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3ed8b-8969-46b7-b939-f38a0c297465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:07.809707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3ed8b-8969-46b7-b939-f38a0c297465 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:07.810362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3ed8b-8969-46b7-b939-f38a0c297465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:07.812380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3ed8b-8969-46b7-b939-f38a0c297465 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:07.845060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 858/1728] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:14:07 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:07.849383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3ed8b-8969-46b7-b939-f38a0c297465 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:07.891902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:07.948459 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ce219-585b-42a6-8edc-c48ce28166a6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:07.984316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3ed8b-8969-46b7-b939-f38a0c297465 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:07.993390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adcba7-9e42-4d7d-ac55-f2beed53a9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:07.993628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adcba7-9e42-4d7d-ac55-f2beed53a9ff admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:07.993712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adcba7-9e42-4d7d-ac55-f2beed53a9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:07.994020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adcba7-9e42-4d7d-ac55-f2beed53a9ff admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:07.994751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0adcba7-9e42-4d7d-ac55-f2beed53a9ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:07.998645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0adcba7-9e42-4d7d-ac55-f2beed53a9ff admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:08.096552 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 859/1729] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:14:07 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:08.099538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adcba7-9e42-4d7d-ac55-f2beed53a9ff admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:08.102993 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a0adcba7-9e42-4d7d-ac55-f2beed53a9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:08.138531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adcba7-9e42-4d7d-ac55-f2beed53a9ff admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:08.143414 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ea3a6-c432-4235-9fec-0bee288343e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:08.143891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ea3a6-c432-4235-9fec-0bee288343e9 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:08.143891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ea3a6-c432-4235-9fec-0bee288343e9 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:08.144575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ea3a6-c432-4235-9fec-0bee288343e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:08.159040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c6906-c629-4793-b8d3-b30734c8d1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:08.162795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c6906-c629-4793-b8d3-b30734c8d1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:08.163072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c6906-c629-4793-b8d3-b30734c8d1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:08.164964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c6906-c629-4793-b8d3-b30734c8d1d3 admin admin] PATH_INFO: `/v3/projects/8ba0f5f714694c249c785253abd18dc8` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:08.168768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c6906-c629-4793-b8d3-b30734c8d1d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba0f5f714694c249c785253abd18dc8)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:08.173260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c6906-c629-4793-b8d3-b30734c8d1d3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:08.299425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ea3a6-c432-4235-9fec-0bee288343e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:08.303510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ea3a6-c432-4235-9fec-0bee288343e9 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:08.325767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 871/1730] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:14:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:14:08.332184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ea3a6-c432-4235-9fec-0bee288343e9 service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:08.347900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-188c6906-c629-4793-b8d3-b30734c8d1d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba0f5f714694c249c785253abd18dc8', 'request_id': 'req-188c6906-c629-4793-b8d3-b30734c8d1d3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:08.378055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ea3a6-c432-4235-9fec-0bee288343e9 service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:08.380210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-188c6906-c629-4793-b8d3-b30734c8d1d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba0f5f714694c249c785253abd18dc8', 'request_id': 'req-188c6906-c629-4793-b8d3-b30734c8d1d3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:08.406317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-188c6906-c629-4793-b8d3-b30734c8d1d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba0f5f714694c249c785253abd18dc8', 'request_id': 'req-188c6906-c629-4793-b8d3-b30734c8d1d3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:08.458629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-188c6906-c629-4793-b8d3-b30734c8d1d3 admin admin] The token cache is being invalidate because project 8ba0f5f714694c249c785253abd18dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:14:08.461632 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-188c6906-c629-4793-b8d3-b30734c8d1d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:08.481772 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-188c6906-c629-4793-b8d3-b30734c8d1d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:08.492476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 860/1731] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:14:08 2022] DELETE /identity/v3/projects/8ba0f5f714694c249c785253abd18dc8 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:08.496911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de96fa2-943f-4779-9e3f-f7866f6fa408 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:08.498950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de96fa2-943f-4779-9e3f-f7866f6fa408 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:08.499455 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de96fa2-943f-4779-9e3f-f7866f6fa408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:08.676030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1de96fa2-943f-4779-9e3f-f7866f6fa408 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:08.685803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1de96fa2-943f-4779-9e3f-f7866f6fa408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:08.813350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b55f1-ca4b-4bf9-a174-38e471f57716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:08.815152 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b55f1-ca4b-4bf9-a174-38e471f57716 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:08.829369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b55f1-ca4b-4bf9-a174-38e471f57716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:08.830210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b55f1-ca4b-4bf9-a174-38e471f57716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:08.831804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b55f1-ca4b-4bf9-a174-38e471f57716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:08.838242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b55f1-ca4b-4bf9-a174-38e471f57716 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:08.882427 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:08.919975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 872/1732] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 592 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:08.937223 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b55f1-ca4b-4bf9-a174-38e471f57716 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:08.949398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-492b55f1-ca4b-4bf9-a174-38e471f57716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:09.115925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b55f1-ca4b-4bf9-a174-38e471f57716 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:09.187158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 861/1733] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 699 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:09.208521 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ef16d-9ed5-411f-b59f-a157c39f9849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:09.209453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ef16d-9ed5-411f-b59f-a157c39f9849 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:09.209865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ef16d-9ed5-411f-b59f-a157c39f9849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:09.210271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ef16d-9ed5-411f-b59f-a157c39f9849 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:09.213711 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e31a34-6543-469a-82e8-5c7daf9d3541 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:09.214245 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e31a34-6543-469a-82e8-5c7daf9d3541 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:09.214709 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e31a34-6543-469a-82e8-5c7daf9d3541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:09.228775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ef16d-9ed5-411f-b59f-a157c39f9849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:09.231920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ef16d-9ed5-411f-b59f-a157c39f9849 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:09.357521 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 873/1734] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:09.359109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ef16d-9ed5-411f-b59f-a157c39f9849 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:09.406119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-35e31a34-6543-469a-82e8-5c7daf9d3541 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:09.417774 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-35e31a34-6543-469a-82e8-5c7daf9d3541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:09.452599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ef16d-9ed5-411f-b59f-a157c39f9849 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:09.499868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f66881-82d0-4f65-a0ad-d53f820da40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:09.501218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f66881-82d0-4f65-a0ad-d53f820da40c admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:09.501683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f66881-82d0-4f65-a0ad-d53f820da40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:09.502188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f66881-82d0-4f65-a0ad-d53f820da40c admin admin] PATH_INFO: `/v3/projects/41d2388a72d14e1fb01aa4bbffc95fa5/users/5262412da2f14cd5b53bcad1b936a4bb/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:09.512721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 862/1735] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:09.515888 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e31a34-6543-469a-82e8-5c7daf9d3541 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:09.542229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:09.603055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e31a34-6543-469a-82e8-5c7daf9d3541 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:09.659442 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e33f2-f244-40f6-84e1-174475c3a39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:09.660482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e33f2-f244-40f6-84e1-174475c3a39b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:09.664248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f66881-82d0-4f65-a0ad-d53f820da40c admin admin] RBAC: Authorizing `identity:create_grant(project_id=41d2388a72d14e1fb01aa4bbffc95fa5, user_id=5262412da2f14cd5b53bcad1b936a4bb, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:09.670539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e33f2-f244-40f6-84e1-174475c3a39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:09.670948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e33f2-f244-40f6-84e1-174475c3a39b admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:09.676636 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e33f2-f244-40f6-84e1-174475c3a39b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:14:09.677190 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e33f2-f244-40f6-84e1-174475c3a39b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:09.701350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f66881-82d0-4f65-a0ad-d53f820da40c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:09.704641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e33f2-f244-40f6-84e1-174475c3a39b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:09.803804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 874/1736] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:09 2022] PUT /identity/v3/projects/41d2388a72d14e1fb01aa4bbffc95fa5/users/5262412da2f14cd5b53bcad1b936a4bb/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:09.835793 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f346-cfe9-4eec-b769-d55c9393a067 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:09.838406 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f346-cfe9-4eec-b769-d55c9393a067 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:09.840577 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f346-cfe9-4eec-b769-d55c9393a067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:09.853648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 863/1737] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:14:09.853648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e33f2-f244-40f6-84e1-174475c3a39b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:09.909513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e33f2-f244-40f6-84e1-174475c3a39b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:10.061229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b791f346-cfe9-4eec-b769-d55c9393a067 None None] MFA Rules not processed for user `5262412da2f14cd5b53bcad1b936a4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:10.063899 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b791f346-cfe9-4eec-b769-d55c9393a067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:10.284597 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f80ba-c1a0-4c8f-804d-6b427e112456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:10.285276 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f80ba-c1a0-4c8f-804d-6b427e112456 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:10.285555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f80ba-c1a0-4c8f-804d-6b427e112456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:10.289198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f80ba-c1a0-4c8f-804d-6b427e112456 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:10.290570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f80ba-c1a0-4c8f-804d-6b427e112456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:10.315469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f80ba-c1a0-4c8f-804d-6b427e112456 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:10.494445 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 864/1738] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:14:09 2022] POST /identity/v3/projects => generated 375 bytes in 642 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:10.500650 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f80ba-c1a0-4c8f-804d-6b427e112456 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:10.564258 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 875/1739] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 732 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:10.584170 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791f346-cfe9-4eec-b769-d55c9393a067 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:10.651217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791f346-cfe9-4eec-b769-d55c9393a067 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:10.750337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f80ba-c1a0-4c8f-804d-6b427e112456 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:10.771231 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133d762-a3d9-45ce-a470-a3961f6bebac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:10.771900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133d762-a3d9-45ce-a470-a3961f6bebac admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:10.772176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133d762-a3d9-45ce-a470-a3961f6bebac admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:10.772450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133d762-a3d9-45ce-a470-a3961f6bebac admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:10.773223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133d762-a3d9-45ce-a470-a3961f6bebac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:10.791567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133d762-a3d9-45ce-a470-a3961f6bebac admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:10.833755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 865/1740] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:14:10 2022] POST /identity/v3/users => generated 288 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:10.838470 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bad8f4-6e72-4265-af31-3c4fe07b0f11 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:10.838969 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bad8f4-6e72-4265-af31-3c4fe07b0f11 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:10.840377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bad8f4-6e72-4265-af31-3c4fe07b0f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:10.903110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745d269-0a61-4113-8250-4a860a85be8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:10.904150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745d269-0a61-4113-8250-4a860a85be8c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:10.904588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745d269-0a61-4113-8250-4a860a85be8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:10.905036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745d269-0a61-4113-8250-4a860a85be8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:10.906032 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745d269-0a61-4113-8250-4a860a85be8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:10.909030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745d269-0a61-4113-8250-4a860a85be8c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:10.925296 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-90bad8f4-6e72-4265-af31-3c4fe07b0f11 None None] MFA Rules not processed for user `d5a45dec40844877a6b5b301fc44a110`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:10.928451 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-90bad8f4-6e72-4265-af31-3c4fe07b0f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:11.027649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 876/1741] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:14:10 2022] POST /identity/v3/projects => generated 377 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:11.029678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745d269-0a61-4113-8250-4a860a85be8c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:11.045233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:11.108306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 866/1742] 10.222.0.73 () {58 vars in 1015 bytes} [Tue Sep 6 21:14:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:11.110317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bad8f4-6e72-4265-af31-3c4fe07b0f11 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:11.156969 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bad8f4-6e72-4265-af31-3c4fe07b0f11 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:11.285947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745d269-0a61-4113-8250-4a860a85be8c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:11.324497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb87fe-4170-4c3f-b242-386fa19836a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:11.330076 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb87fe-4170-4c3f-b242-386fa19836a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:11.330372 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb87fe-4170-4c3f-b242-386fa19836a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:11.330696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb87fe-4170-4c3f-b242-386fa19836a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:11.331451 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb87fe-4170-4c3f-b242-386fa19836a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:11.339497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb87fe-4170-4c3f-b242-386fa19836a8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:11.361736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:11.418558 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 877/1743] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:11.419126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb87fe-4170-4c3f-b242-386fa19836a8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:11.429626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115be68-a347-4bb4-acaf-245aac9d04e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:11.431262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115be68-a347-4bb4-acaf-245aac9d04e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:11.431878 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115be68-a347-4bb4-acaf-245aac9d04e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:11.432065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-effb87fe-4170-4c3f-b242-386fa19836a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:11.433073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115be68-a347-4bb4-acaf-245aac9d04e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:11.434651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115be68-a347-4bb4-acaf-245aac9d04e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:11.437101 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115be68-a347-4bb4-acaf-245aac9d04e7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:11.496982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 867/1744] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:14:11 2022] POST /identity/v3/users => generated 295 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:11.500883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115be68-a347-4bb4-acaf-245aac9d04e7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:11.561265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115be68-a347-4bb4-acaf-245aac9d04e7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:11.586524 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81086f7f-4030-40b7-bdb8-86b5e9a07092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:11.588883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81086f7f-4030-40b7-bdb8-86b5e9a07092 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:11.591085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81086f7f-4030-40b7-bdb8-86b5e9a07092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:11.591868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81086f7f-4030-40b7-bdb8-86b5e9a07092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:11.593209 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81086f7f-4030-40b7-bdb8-86b5e9a07092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:11.601577 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81086f7f-4030-40b7-bdb8-86b5e9a07092 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:11.636990 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb87fe-4170-4c3f-b242-386fa19836a8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:11.650247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e70212-2b82-437f-a810-1b1bd5bf5fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:11.654083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 868/1745] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:11.656261 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81086f7f-4030-40b7-bdb8-86b5e9a07092 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:11.657633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e70212-2b82-437f-a810-1b1bd5bf5fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:11.658022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e70212-2b82-437f-a810-1b1bd5bf5fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:11.658317 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e70212-2b82-437f-a810-1b1bd5bf5fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:11.696820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e70212-2b82-437f-a810-1b1bd5bf5fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:11.696820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e70212-2b82-437f-a810-1b1bd5bf5fe4 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:11.698364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81086f7f-4030-40b7-bdb8-86b5e9a07092 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:11.723372 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 878/1746] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:11.725163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e70212-2b82-437f-a810-1b1bd5bf5fe4 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:11.725447 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce63b9-021c-4f1e-a94c-c660bdaf7db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:11.726211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce63b9-021c-4f1e-a94c-c660bdaf7db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:11.726489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce63b9-021c-4f1e-a94c-c660bdaf7db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:11.726759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce63b9-021c-4f1e-a94c-c660bdaf7db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:11.727524 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce63b9-021c-4f1e-a94c-c660bdaf7db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:11.731598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce63b9-021c-4f1e-a94c-c660bdaf7db2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:11.765020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e70212-2b82-437f-a810-1b1bd5bf5fe4 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:11.774276 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 869/1747] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:11.775449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce63b9-021c-4f1e-a94c-c660bdaf7db2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:11.783770 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d066f-2669-4551-8407-df725f84aacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:11.786563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d066f-2669-4551-8407-df725f84aacb admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:11.786847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d066f-2669-4551-8407-df725f84aacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:11.787123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d066f-2669-4551-8407-df725f84aacb admin admin] PATH_INFO: `/v3/projects/4674191824504af487e97ada3bd6c17b/users/b31347907161449b806aa08c49565869/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:11.805236 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce63b9-021c-4f1e-a94c-c660bdaf7db2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:11.833367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ced36-7109-4e11-8c4a-008d0956061c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:11.834823 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ced36-7109-4e11-8c4a-008d0956061c admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:11.836264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ced36-7109-4e11-8c4a-008d0956061c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:11.837199 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ced36-7109-4e11-8c4a-008d0956061c admin admin] PATH_INFO: `/v3/projects/c25d40777cde4bfa892b4a87cdb472f5/users/ca1a15334d7946f2b049c0cedd9f0456/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:11.857933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d066f-2669-4551-8407-df725f84aacb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4674191824504af487e97ada3bd6c17b, user_id=b31347907161449b806aa08c49565869, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:11.874659 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d066f-2669-4551-8407-df725f84aacb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:11.890032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ced36-7109-4e11-8c4a-008d0956061c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c25d40777cde4bfa892b4a87cdb472f5, user_id=ca1a15334d7946f2b049c0cedd9f0456, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:11.899902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ced36-7109-4e11-8c4a-008d0956061c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:11.929647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 879/1748] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:11 2022] PUT /identity/v3/projects/4674191824504af487e97ada3bd6c17b/users/b31347907161449b806aa08c49565869/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:11.942976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d66b9-4a8a-41b0-9dae-a45065c2eb14 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:11.943616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d66b9-4a8a-41b0-9dae-a45065c2eb14 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:11.945046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d66b9-4a8a-41b0-9dae-a45065c2eb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:11.962225 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 870/1749] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:11 2022] PUT /identity/v3/projects/c25d40777cde4bfa892b4a87cdb472f5/users/ca1a15334d7946f2b049c0cedd9f0456/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:11.972517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ced36-7109-4e11-8c4a-008d0956061c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:12.017000 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-019d66b9-4a8a-41b0-9dae-a45065c2eb14 None None] MFA Rules not processed for user `b31347907161449b806aa08c49565869`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:12.017367 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-019d66b9-4a8a-41b0-9dae-a45065c2eb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:12.217213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ced36-7109-4e11-8c4a-008d0956061c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:12.239215 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5e562-ecb8-46e5-9e30-3972121e9f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:12.239868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5e562-ecb8-46e5-9e30-3972121e9f3e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:12.240201 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5e562-ecb8-46e5-9e30-3972121e9f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:12.240478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5e562-ecb8-46e5-9e30-3972121e9f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:12.241156 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5e562-ecb8-46e5-9e30-3972121e9f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:12.243644 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5e562-ecb8-46e5-9e30-3972121e9f3e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:12.271300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 871/1750] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:14:12.275888 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5e562-ecb8-46e5-9e30-3972121e9f3e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:12.290493 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 880/1751] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:14:12.311540 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5e562-ecb8-46e5-9e30-3972121e9f3e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:12.322893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92516971-5f28-493f-adc9-9d1b72b1ad4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:12.332803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92516971-5f28-493f-adc9-9d1b72b1ad4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:12.333028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92516971-5f28-493f-adc9-9d1b72b1ad4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:12.333293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92516971-5f28-493f-adc9-9d1b72b1ad4f admin admin] PATH_INFO: `/v3/projects/c25d40777cde4bfa892b4a87cdb472f5/users/ca1a15334d7946f2b049c0cedd9f0456/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:12.340159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92516971-5f28-493f-adc9-9d1b72b1ad4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c25d40777cde4bfa892b4a87cdb472f5, user_id=ca1a15334d7946f2b049c0cedd9f0456, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:12.346963 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92516971-5f28-493f-adc9-9d1b72b1ad4f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:12.390227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 872/1752] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:12 2022] PUT /identity/v3/projects/c25d40777cde4bfa892b4a87cdb472f5/users/ca1a15334d7946f2b049c0cedd9f0456/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:12.400969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d4753-f1a4-4ab7-9db8-583c9cf20706 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:12.401722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d4753-f1a4-4ab7-9db8-583c9cf20706 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:12.402003 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d4753-f1a4-4ab7-9db8-583c9cf20706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:12.475363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ab9d4753-f1a4-4ab7-9db8-583c9cf20706 None None] MFA Rules not processed for user `ca1a15334d7946f2b049c0cedd9f0456`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:12.483602 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ab9d4753-f1a4-4ab7-9db8-583c9cf20706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:12.640814 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:12.694818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 881/1753] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:12.706864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92516971-5f28-493f-adc9-9d1b72b1ad4f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:12.714273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-92516971-5f28-493f-adc9-9d1b72b1ad4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:12.781708 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:12.903699 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92516971-5f28-493f-adc9-9d1b72b1ad4f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:12.915447 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7092477-e413-404b-8dc3-a3d97d32956a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:12.916108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7092477-e413-404b-8dc3-a3d97d32956a service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:12.916386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7092477-e413-404b-8dc3-a3d97d32956a service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:12.916916 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7092477-e413-404b-8dc3-a3d97d32956a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:13.064043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d4753-f1a4-4ab7-9db8-583c9cf20706 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:13.065643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ab9d4753-f1a4-4ab7-9db8-583c9cf20706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:13.118452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7092477-e413-404b-8dc3-a3d97d32956a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:13.124787 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7092477-e413-404b-8dc3-a3d97d32956a service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:13.158474 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 873/1754] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:13.298060 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d4753-f1a4-4ab7-9db8-583c9cf20706 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:13.317555 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a480a76-6239-4c53-aa5f-06ffa5c62771 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:13.318277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a480a76-6239-4c53-aa5f-06ffa5c62771 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:13.318743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a480a76-6239-4c53-aa5f-06ffa5c62771 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:13.319182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a480a76-6239-4c53-aa5f-06ffa5c62771 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:13.503097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a480a76-6239-4c53-aa5f-06ffa5c62771 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:13.504866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a480a76-6239-4c53-aa5f-06ffa5c62771 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:13.553545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 882/1755] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:14:13.918280 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7092477-e413-404b-8dc3-a3d97d32956a service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:13.919765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e7092477-e413-404b-8dc3-a3d97d32956a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:14.158222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7092477-e413-404b-8dc3-a3d97d32956a service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:14.169664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:14.170399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:14.170685 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:14.171122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] PATH_INFO: `/v3/users/f63913fa87f240ce8c39190d51a4b9de` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:14.172357 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f63913fa87f240ce8c39190d51a4b9de)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:14.175926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:14.344077 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f63913fa87f240ce8c39190d51a4b9de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:14.347233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f63913fa87f240ce8c39190d51a4b9de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:14.370000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f63913fa87f240ce8c39190d51a4b9de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:14.396526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:14.410917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f63913fa87f240ce8c39190d51a4b9de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:14.412105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f63913fa87f240ce8c39190d51a4b9de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:14.436999 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 874/1756] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:14:13 2022] DELETE /identity/v3/users/f63913fa87f240ce8c39190d51a4b9de => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:14.438430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10620f5-dd18-45a8-ba56-42a791abb0d8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:14.438956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10620f5-dd18-45a8-ba56-42a791abb0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:14.438956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10620f5-dd18-45a8-ba56-42a791abb0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:14.477688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f10620f5-dd18-45a8-ba56-42a791abb0d8 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:14.479051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f10620f5-dd18-45a8-ba56-42a791abb0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:14.682231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 883/1757] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:14.723835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:14.725770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:14.866869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10620f5-dd18-45a8-ba56-42a791abb0d8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:14.869364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f10620f5-dd18-45a8-ba56-42a791abb0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:14.889505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10620f5-dd18-45a8-ba56-42a791abb0d8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:14.905419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:14.906491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:14.907171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:14.907623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] PATH_INFO: `/v3/users/b31347907161449b806aa08c49565869` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:14.909297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b31347907161449b806aa08c49565869)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:14.911789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6205c98-2182-49cd-8b4e-e1b97cda24fc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:14.914167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:14.915758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffe083-b02f-44cc-bbcf-2291a3080574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:14.916408 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffe083-b02f-44cc-bbcf-2291a3080574 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:14.916797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffe083-b02f-44cc-bbcf-2291a3080574 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:14.917081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffe083-b02f-44cc-bbcf-2291a3080574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:14.935875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ffe083-b02f-44cc-bbcf-2291a3080574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:14.959577 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ffe083-b02f-44cc-bbcf-2291a3080574 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:14.977921 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:15.012965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 875/1758] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:15.045203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b31347907161449b806aa08c49565869'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:15.046259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b31347907161449b806aa08c49565869'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:15.070937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b31347907161449b806aa08c49565869'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:15.093465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b31347907161449b806aa08c49565869'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:15.110665 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b31347907161449b806aa08c49565869'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:15.138536 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 884/1759] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:14:14 2022] DELETE /identity/v3/users/b31347907161449b806aa08c49565869 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:15.143866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3bb60-3f0b-4ff0-89f1-a167ae78d99d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:15.144243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3bb60-3f0b-4ff0-89f1-a167ae78d99d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:15.146668 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3bb60-3f0b-4ff0-89f1-a167ae78d99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:15.199369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ebe3bb60-3f0b-4ff0-89f1-a167ae78d99d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:15.205460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ebe3bb60-3f0b-4ff0-89f1-a167ae78d99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:15.320376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 876/1760] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:15.469925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:15.471390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:15.660234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2151a3d-ca75-4eae-8507-8411edef8a92 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:15.673072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e301a51-42b1-4fc9-a42a-895792067a21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:15.674401 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301a51-42b1-4fc9-a42a-895792067a21 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:15.675245 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301a51-42b1-4fc9-a42a-895792067a21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:15.675729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301a51-42b1-4fc9-a42a-895792067a21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:15.703696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e301a51-42b1-4fc9-a42a-895792067a21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:15.705715 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e301a51-42b1-4fc9-a42a-895792067a21 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:15.733350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 885/1761] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:16.911479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f703b0-48fb-4c2c-8061-aaa4f038167f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:16.912091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f703b0-48fb-4c2c-8061-aaa4f038167f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:16.913775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f703b0-48fb-4c2c-8061-aaa4f038167f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:16.914631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 877/1762] 10.222.0.79 () {54 vars in 921 bytes} [Tue Sep 6 21:14:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:14:16.941307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa56e4b-ef73-4969-b997-0e8e419976ae None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:16.941824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa56e4b-ef73-4969-b997-0e8e419976ae None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:16.941824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa56e4b-ef73-4969-b997-0e8e419976ae None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:16.943201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 886/1763] 10.222.0.79 () {54 vars in 921 bytes} [Tue Sep 6 21:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:14:16.991461 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1754383-adbc-480a-9e74-70b5374d99d1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:16.991770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1754383-adbc-480a-9e74-70b5374d99d1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:16.992038 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1754383-adbc-480a-9e74-70b5374d99d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:17.049534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e1754383-adbc-480a-9e74-70b5374d99d1 None None] MFA Rules not processed for user `12d51454d20c46d7a5e6d963a2f987cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:17.055841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e1754383-adbc-480a-9e74-70b5374d99d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:17.063651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a08f84-aa0b-4b9f-addc-9d658bb3310e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:17.063943 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a08f84-aa0b-4b9f-addc-9d658bb3310e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:17.064218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a08f84-aa0b-4b9f-addc-9d658bb3310e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:17.128402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b8a08f84-aa0b-4b9f-addc-9d658bb3310e None None] MFA Rules not processed for user `12d51454d20c46d7a5e6d963a2f987cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:17.130939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b8a08f84-aa0b-4b9f-addc-9d658bb3310e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:17.288762 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 878/1764] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Sep 6 21:14:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:17.308480 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 887/1765] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Sep 6 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:17.585656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1754383-adbc-480a-9e74-70b5374d99d1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:17.585656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e1754383-adbc-480a-9e74-70b5374d99d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:17.601186 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1754383-adbc-480a-9e74-70b5374d99d1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:17.607154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a17629-7910-4406-be1b-4e29cc1347fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:17.607955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a17629-7910-4406-be1b-4e29cc1347fa service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:17.608254 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a17629-7910-4406-be1b-4e29cc1347fa service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:17.608531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a17629-7910-4406-be1b-4e29cc1347fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:17.620074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a17629-7910-4406-be1b-4e29cc1347fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:17.622120 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a17629-7910-4406-be1b-4e29cc1347fa service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:17.635539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 879/1766] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:17.737829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a08f84-aa0b-4b9f-addc-9d658bb3310e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:17.750963 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a08f84-aa0b-4b9f-addc-9d658bb3310e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:17.757829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713635a-128a-4f0e-be4c-7c9f27b4048d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:17.758928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713635a-128a-4f0e-be4c-7c9f27b4048d service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:17.759414 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713635a-128a-4f0e-be4c-7c9f27b4048d service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:17.759872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713635a-128a-4f0e-be4c-7c9f27b4048d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:17.762440 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a17629-7910-4406-be1b-4e29cc1347fa service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:17.763832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-42a17629-7910-4406-be1b-4e29cc1347fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:17.793022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a17629-7910-4406-be1b-4e29cc1347fa service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:17.793022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:17.793022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:17.793022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:17.793424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56 admin admin] PATH_INFO: `/v3/projects/4674191824504af487e97ada3bd6c17b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:17.793424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4674191824504af487e97ada3bd6c17b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:17.793424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:17.793870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:17.806679 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713635a-128a-4f0e-be4c-7c9f27b4048d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:17.811183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713635a-128a-4f0e-be4c-7c9f27b4048d service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:17.830845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 888/1767] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:17.867377 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713635a-128a-4f0e-be4c-7c9f27b4048d service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:17.869316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0713635a-128a-4f0e-be4c-7c9f27b4048d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:17.886030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713635a-128a-4f0e-be4c-7c9f27b4048d service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:17.907018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4674191824504af487e97ada3bd6c17b', 'request_id': 'req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:17.911528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:17.931440 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4674191824504af487e97ada3bd6c17b', 'request_id': 'req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:17.945181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4674191824504af487e97ada3bd6c17b', 'request_id': 'req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:17.975482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56 admin admin] The token cache is being invalidate because project 4674191824504af487e97ada3bd6c17b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:14:17.976080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:17.977642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14ba30dd-2af3-4462-be7e-8dfb4fbafd56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:17.979889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 880/1768] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:14:17 2022] DELETE /identity/v3/projects/4674191824504af487e97ada3bd6c17b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:14:17.984536 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:17.985478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:17.985630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c49b40-d477-4c51-9861-aac7a05bdf22 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:17.985936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:17.986074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c49b40-d477-4c51-9861-aac7a05bdf22 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:17.986515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] PATH_INFO: `/v3/projects/4c5895bb13774751b365bfcfea2ba68d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:17.986791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c49b40-d477-4c51-9861-aac7a05bdf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:17.988304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5895bb13774751b365bfcfea2ba68d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:17.990449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:18.018964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-51c49b40-d477-4c51-9861-aac7a05bdf22 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:18.021019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-51c49b40-d477-4c51-9861-aac7a05bdf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:18.032854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 881/1769] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:18.041707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb7e96-aaa2-415d-a89c-1c61a0d1d348 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:18.043967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb7e96-aaa2-415d-a89c-1c61a0d1d348 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:18.044348 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb7e96-aaa2-415d-a89c-1c61a0d1d348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:18.090841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5895bb13774751b365bfcfea2ba68d', 'request_id': 'req-7e5b52e4-78b0-4316-af7c-d792f943315f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:18.093021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5fbb7e96-aaa2-415d-a89c-1c61a0d1d348 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:18.095980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5fbb7e96-aaa2-415d-a89c-1c61a0d1d348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:18.100393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5895bb13774751b365bfcfea2ba68d', 'request_id': 'req-7e5b52e4-78b0-4316-af7c-d792f943315f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:18.110542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5895bb13774751b365bfcfea2ba68d', 'request_id': 'req-7e5b52e4-78b0-4316-af7c-d792f943315f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:18.140683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] The token cache is being invalidate because project 4c5895bb13774751b365bfcfea2ba68d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:14:18.140995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:18.141727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:18.144075 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 889/1770] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:14:17 2022] DELETE /identity/v3/projects/4c5895bb13774751b365bfcfea2ba68d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:14:18.147823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:18.149562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:18.188201 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 882/1771] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:18.192762 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb7e96-aaa2-415d-a89c-1c61a0d1d348 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:18.202822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb7e96-aaa2-415d-a89c-1c61a0d1d348 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:18.207995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d735d08-7fde-44aa-9112-be6f2bb7f985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:18.208614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d735d08-7fde-44aa-9112-be6f2bb7f985 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:18.208986 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d735d08-7fde-44aa-9112-be6f2bb7f985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:18.209263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d735d08-7fde-44aa-9112-be6f2bb7f985 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:18.210064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d735d08-7fde-44aa-9112-be6f2bb7f985 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:14:18.210370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d735d08-7fde-44aa-9112-be6f2bb7f985 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:18.211976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d735d08-7fde-44aa-9112-be6f2bb7f985 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:18.228712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 883/1772] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:14:18.233386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d735d08-7fde-44aa-9112-be6f2bb7f985 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:18.237722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b52e4-78b0-4316-af7c-d792f943315f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:18.243181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:18.243471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d735d08-7fde-44aa-9112-be6f2bb7f985 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:18.243851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:18.244133 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:18.244467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] PATH_INFO: `/v3/users/2bc9f2a7ace943a09185a2f7f83bb80b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:18.245562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bc9f2a7ace943a09185a2f7f83bb80b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:18.247566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:18.248716 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed27775-00f7-46d9-bc24-e9320b33437b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:18.249522 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed27775-00f7-46d9-bc24-e9320b33437b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:18.249797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed27775-00f7-46d9-bc24-e9320b33437b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:18.250450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed27775-00f7-46d9-bc24-e9320b33437b admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:18.251078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed27775-00f7-46d9-bc24-e9320b33437b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:18.253084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed27775-00f7-46d9-bc24-e9320b33437b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:18.294257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 884/1773] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:14:18 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:14:18.296644 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed27775-00f7-46d9-bc24-e9320b33437b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:18.327028 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bc9f2a7ace943a09185a2f7f83bb80b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:18.327968 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bc9f2a7ace943a09185a2f7f83bb80b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:18.333478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:18.339810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bc9f2a7ace943a09185a2f7f83bb80b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:18.355952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bc9f2a7ace943a09185a2f7f83bb80b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:18.356817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bc9f2a7ace943a09185a2f7f83bb80b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:18.367857 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:18.381393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 890/1774] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:14:18 2022] DELETE /identity/v3/users/2bc9f2a7ace943a09185a2f7f83bb80b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:14:18.387088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed27775-00f7-46d9-bc24-e9320b33437b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:18.488063 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df464866-c957-48ce-b94c-f3f5e9fa817e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:18.488883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df464866-c957-48ce-b94c-f3f5e9fa817e admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:18.489161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df464866-c957-48ce-b94c-f3f5e9fa817e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:18.489593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df464866-c957-48ce-b94c-f3f5e9fa817e admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:18.490394 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df464866-c957-48ce-b94c-f3f5e9fa817e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:18.492547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df464866-c957-48ce-b94c-f3f5e9fa817e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:18.523771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 885/1775] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:14:18 2022] POST /identity/v3/users => generated 283 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:18.528167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:18.529920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:18.543676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a23d78-dd16-4dc4-8a73-15aaa362a212 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:18.548516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35642d45-a235-43b7-83c7-88d3c1dd584e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:18.549165 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35642d45-a235-43b7-83c7-88d3c1dd584e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:18.550784 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35642d45-a235-43b7-83c7-88d3c1dd584e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:18.551069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35642d45-a235-43b7-83c7-88d3c1dd584e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:18.551725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35642d45-a235-43b7-83c7-88d3c1dd584e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:18.553393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35642d45-a235-43b7-83c7-88d3c1dd584e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:18.564534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 891/1776] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:18.568876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df464866-c957-48ce-b94c-f3f5e9fa817e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:18.580223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df464866-c957-48ce-b94c-f3f5e9fa817e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:18.586176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3b616-d513-4919-9776-180e74a0c6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:18.586909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3b616-d513-4919-9776-180e74a0c6ea admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:18.587254 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3b616-d513-4919-9776-180e74a0c6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:18.587603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3b616-d513-4919-9776-180e74a0c6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:18.588328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3b616-d513-4919-9776-180e74a0c6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:18.590143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3b616-d513-4919-9776-180e74a0c6ea admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:18.604641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 886/1777] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:18.609130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35642d45-a235-43b7-83c7-88d3c1dd584e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:18.622395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35642d45-a235-43b7-83c7-88d3c1dd584e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:18.627801 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f9fd7-974d-4114-934e-64e8be647412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:18.628508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f9fd7-974d-4114-934e-64e8be647412 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:18.628812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f9fd7-974d-4114-934e-64e8be647412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:18.629104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f9fd7-974d-4114-934e-64e8be647412 admin admin] PATH_INFO: `/v3/projects/5f6d5fe8091b4954be27a235d286089d/users/114eaad788dd479a8cf92a7e0eb18ff2/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:18.647204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f9fd7-974d-4114-934e-64e8be647412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f6d5fe8091b4954be27a235d286089d, user_id=114eaad788dd479a8cf92a7e0eb18ff2, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:18.649863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f9fd7-974d-4114-934e-64e8be647412 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:18.667319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 892/1778] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:18 2022] PUT /identity/v3/projects/5f6d5fe8091b4954be27a235d286089d/users/114eaad788dd479a8cf92a7e0eb18ff2/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:18.678618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3707f6-f6d2-4130-afa9-86222c45a134 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:18.678938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3707f6-f6d2-4130-afa9-86222c45a134 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:18.679198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3707f6-f6d2-4130-afa9-86222c45a134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:18.713091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9b3707f6-f6d2-4130-afa9-86222c45a134 None None] MFA Rules not processed for user `114eaad788dd479a8cf92a7e0eb18ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:18.714988 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9b3707f6-f6d2-4130-afa9-86222c45a134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:18.803517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 887/1779] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:18.808089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f9fd7-974d-4114-934e-64e8be647412 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:18.854567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3707f6-f6d2-4130-afa9-86222c45a134 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:18.855982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9b3707f6-f6d2-4130-afa9-86222c45a134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:18.881091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f9fd7-974d-4114-934e-64e8be647412 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:18.885710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494bbfe-fc5f-4e51-a50c-d71673ac0b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:18.886310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494bbfe-fc5f-4e51-a50c-d71673ac0b61 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:18.886585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494bbfe-fc5f-4e51-a50c-d71673ac0b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:18.886850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494bbfe-fc5f-4e51-a50c-d71673ac0b61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:18.887850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494bbfe-fc5f-4e51-a50c-d71673ac0b61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:18.889461 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494bbfe-fc5f-4e51-a50c-d71673ac0b61 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:18.930909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 893/1780] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:14:18 2022] POST /identity/v3/projects => generated 363 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:18.935053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494bbfe-fc5f-4e51-a50c-d71673ac0b61 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:18.957864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3707f6-f6d2-4130-afa9-86222c45a134 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:18.966765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:19.008781 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494bbfe-fc5f-4e51-a50c-d71673ac0b61 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.013660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97b535-2888-4f5d-9e58-4abab29aa83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.014280 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97b535-2888-4f5d-9e58-4abab29aa83b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.014563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97b535-2888-4f5d-9e58-4abab29aa83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.014841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97b535-2888-4f5d-9e58-4abab29aa83b admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.015512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97b535-2888-4f5d-9e58-4abab29aa83b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.017494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97b535-2888-4f5d-9e58-4abab29aa83b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.017975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117522dd-ca57-45b7-b075-6c112ffefd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.018650 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117522dd-ca57-45b7-b075-6c112ffefd9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.018930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117522dd-ca57-45b7-b075-6c112ffefd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.019208 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117522dd-ca57-45b7-b075-6c112ffefd9c admin admin] PATH_INFO: `/v3/projects/d7b17da3dd5f49169e3c9f6640ea384a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.020325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117522dd-ca57-45b7-b075-6c112ffefd9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7b17da3dd5f49169e3c9f6640ea384a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.022325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117522dd-ca57-45b7-b075-6c112ffefd9c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.040173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 894/1781] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:14:18 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:14:19.044059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97b535-2888-4f5d-9e58-4abab29aa83b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.054396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97b535-2888-4f5d-9e58-4abab29aa83b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.059212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb65b4-c646-4d1c-a384-9e60aadf2494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.059778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb65b4-c646-4d1c-a384-9e60aadf2494 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.060055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb65b4-c646-4d1c-a384-9e60aadf2494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.060325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb65b4-c646-4d1c-a384-9e60aadf2494 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.061006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb65b4-c646-4d1c-a384-9e60aadf2494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.062662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb65b4-c646-4d1c-a384-9e60aadf2494 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.064845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:19.074082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-117522dd-ca57-45b7-b075-6c112ffefd9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7b17da3dd5f49169e3c9f6640ea384a', 'request_id': 'req-117522dd-ca57-45b7-b075-6c112ffefd9c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:19.075805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 895/1782] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:14:19.079203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb65b4-c646-4d1c-a384-9e60aadf2494 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.082443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-117522dd-ca57-45b7-b075-6c112ffefd9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7b17da3dd5f49169e3c9f6640ea384a', 'request_id': 'req-117522dd-ca57-45b7-b075-6c112ffefd9c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:19.089129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-117522dd-ca57-45b7-b075-6c112ffefd9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7b17da3dd5f49169e3c9f6640ea384a', 'request_id': 'req-117522dd-ca57-45b7-b075-6c112ffefd9c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:19.089254 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb65b4-c646-4d1c-a384-9e60aadf2494 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.100935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-117522dd-ca57-45b7-b075-6c112ffefd9c admin admin] The token cache is being invalidate because project d7b17da3dd5f49169e3c9f6640ea384a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:14:19.101227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-117522dd-ca57-45b7-b075-6c112ffefd9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:19.101993 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-117522dd-ca57-45b7-b075-6c112ffefd9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:19.103637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 888/1783] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:14:18 2022] DELETE /identity/v3/projects/d7b17da3dd5f49169e3c9f6640ea384a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:14:19.108375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36020b-2598-4ea5-9a84-db060b44ecc4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.108675 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36020b-2598-4ea5-9a84-db060b44ecc4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.108966 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36020b-2598-4ea5-9a84-db060b44ecc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.128849 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cc36020b-2598-4ea5-9a84-db060b44ecc4 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:19.130108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cc36020b-2598-4ea5-9a84-db060b44ecc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:19.147423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e548800-ed75-4ed4-9ea2-38e884fc04d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.148167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e548800-ed75-4ed4-9ea2-38e884fc04d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.148379 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e548800-ed75-4ed4-9ea2-38e884fc04d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.148668 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e548800-ed75-4ed4-9ea2-38e884fc04d8 admin admin] PATH_INFO: `/v3/projects/b51a2462858348c482e2423d706abd1b/users/f10667a78b2e4aa099efe0fb7e67f387/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.166413 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e548800-ed75-4ed4-9ea2-38e884fc04d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b51a2462858348c482e2423d706abd1b, user_id=f10667a78b2e4aa099efe0fb7e67f387, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.169340 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e548800-ed75-4ed4-9ea2-38e884fc04d8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.184683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 896/1784] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:19 2022] PUT /identity/v3/projects/b51a2462858348c482e2423d706abd1b/users/f10667a78b2e4aa099efe0fb7e67f387/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:14:19.188562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e548800-ed75-4ed4-9ea2-38e884fc04d8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.190260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0e548800-ed75-4ed4-9ea2-38e884fc04d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:19.198209 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 889/1785] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:19.210795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b46b1f-ba38-40b6-a569-6e1cba7f85e5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.211077 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b46b1f-ba38-40b6-a569-6e1cba7f85e5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.211350 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b46b1f-ba38-40b6-a569-6e1cba7f85e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.232185 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b5b46b1f-ba38-40b6-a569-6e1cba7f85e5 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:19.233453 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b5b46b1f-ba38-40b6-a569-6e1cba7f85e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:19.261961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e548800-ed75-4ed4-9ea2-38e884fc04d8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.267581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e22ac3-991a-4dfc-aa7e-e8935e01a678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.268269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e22ac3-991a-4dfc-aa7e-e8935e01a678 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.268558 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e22ac3-991a-4dfc-aa7e-e8935e01a678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.272235 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e22ac3-991a-4dfc-aa7e-e8935e01a678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.272956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e22ac3-991a-4dfc-aa7e-e8935e01a678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.274597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e22ac3-991a-4dfc-aa7e-e8935e01a678 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.275295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:19.290760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 897/1786] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:14:19.292254 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e22ac3-991a-4dfc-aa7e-e8935e01a678 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.298367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 890/1787] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:14:19.302939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b46b1f-ba38-40b6-a569-6e1cba7f85e5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.303835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e22ac3-991a-4dfc-aa7e-e8935e01a678 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.309279 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ead9d-3070-4876-8955-e564efca16d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.309960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ead9d-3070-4876-8955-e564efca16d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.310242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ead9d-3070-4876-8955-e564efca16d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.310515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ead9d-3070-4876-8955-e564efca16d1 admin admin] PATH_INFO: `/v3/projects/b51a2462858348c482e2423d706abd1b/users/f10667a78b2e4aa099efe0fb7e67f387/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.312484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ead9d-3070-4876-8955-e564efca16d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b51a2462858348c482e2423d706abd1b, user_id=f10667a78b2e4aa099efe0fb7e67f387, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.314631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b46b1f-ba38-40b6-a569-6e1cba7f85e5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.315223 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ead9d-3070-4876-8955-e564efca16d1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.320087 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e769b-1f9c-4d29-8855-9e36749e7c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.320781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e769b-1f9c-4d29-8855-9e36749e7c21 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.320972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e769b-1f9c-4d29-8855-9e36749e7c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.321241 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e769b-1f9c-4d29-8855-9e36749e7c21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.321891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e769b-1f9c-4d29-8855-9e36749e7c21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:14:19.322417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e769b-1f9c-4d29-8855-9e36749e7c21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.323774 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e769b-1f9c-4d29-8855-9e36749e7c21 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.329396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 898/1788] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:19 2022] PUT /identity/v3/projects/b51a2462858348c482e2423d706abd1b/users/f10667a78b2e4aa099efe0fb7e67f387/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:19.340027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 891/1789] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:14:19.340977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e769b-1f9c-4d29-8855-9e36749e7c21 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.341147 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3ab8a-322c-4959-82a0-4f5e768dbd9f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.341495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3ab8a-322c-4959-82a0-4f5e768dbd9f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.341974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3ab8a-322c-4959-82a0-4f5e768dbd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.351581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e769b-1f9c-4d29-8855-9e36749e7c21 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.374429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b8a3ab8a-322c-4959-82a0-4f5e768dbd9f None None] MFA Rules not processed for user `f10667a78b2e4aa099efe0fb7e67f387`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:19.375992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b8a3ab8a-322c-4959-82a0-4f5e768dbd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:19.414535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af77448-1b0f-4bc8-ad73-8c0096f01543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.415197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af77448-1b0f-4bc8-ad73-8c0096f01543 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.415541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af77448-1b0f-4bc8-ad73-8c0096f01543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.415878 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af77448-1b0f-4bc8-ad73-8c0096f01543 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.416600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af77448-1b0f-4bc8-ad73-8c0096f01543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.418568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af77448-1b0f-4bc8-ad73-8c0096f01543 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.428899 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:19.448978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 899/1790] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:19.449906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 892/1791] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:14:19 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:19.454099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3ab8a-322c-4959-82a0-4f5e768dbd9f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.460349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af77448-1b0f-4bc8-ad73-8c0096f01543 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.461958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1af77448-1b0f-4bc8-ad73-8c0096f01543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:19.464122 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3ab8a-322c-4959-82a0-4f5e768dbd9f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.522535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af77448-1b0f-4bc8-ad73-8c0096f01543 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.522857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0ab03-4d6f-4126-97d1-735f698b8671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.523484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0ab03-4d6f-4126-97d1-735f698b8671 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.523758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0ab03-4d6f-4126-97d1-735f698b8671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.524022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0ab03-4d6f-4126-97d1-735f698b8671 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.524668 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0ab03-4d6f-4126-97d1-735f698b8671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.526597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0ab03-4d6f-4126-97d1-735f698b8671 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.527953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac6853-10ef-439b-a4cc-453c3c2f8213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.528491 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac6853-10ef-439b-a4cc-453c3c2f8213 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.528813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac6853-10ef-439b-a4cc-453c3c2f8213 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.529193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac6853-10ef-439b-a4cc-453c3c2f8213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.553704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 900/1792] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:14:19 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:19.557721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0ab03-4d6f-4126-97d1-735f698b8671 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.568042 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0ab03-4d6f-4126-97d1-735f698b8671 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.572950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91733469-2fc6-45bf-ad6e-b5132187db14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.573673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91733469-2fc6-45bf-ad6e-b5132187db14 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.573952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91733469-2fc6-45bf-ad6e-b5132187db14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.574224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91733469-2fc6-45bf-ad6e-b5132187db14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.575245 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91733469-2fc6-45bf-ad6e-b5132187db14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.576751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91733469-2fc6-45bf-ad6e-b5132187db14 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.587946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 901/1793] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:19.591902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91733469-2fc6-45bf-ad6e-b5132187db14 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.603159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91733469-2fc6-45bf-ad6e-b5132187db14 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.608568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f2b23-1e3d-4e94-9931-084e2074d539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.609453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f2b23-1e3d-4e94-9931-084e2074d539 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.609807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f2b23-1e3d-4e94-9931-084e2074d539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.610252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f2b23-1e3d-4e94-9931-084e2074d539 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.610903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f2b23-1e3d-4e94-9931-084e2074d539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.612981 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f2b23-1e3d-4e94-9931-084e2074d539 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.615032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac6853-10ef-439b-a4cc-453c3c2f8213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.616312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac6853-10ef-439b-a4cc-453c3c2f8213 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.626294 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 902/1794] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:19.630488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 893/1795] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:14:19.630918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f2b23-1e3d-4e94-9931-084e2074d539 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.642073 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f2b23-1e3d-4e94-9931-084e2074d539 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.647298 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82617070-a841-48df-85bb-970f15d9e60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.648418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82617070-a841-48df-85bb-970f15d9e60f admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.648418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82617070-a841-48df-85bb-970f15d9e60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.648583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82617070-a841-48df-85bb-970f15d9e60f admin admin] PATH_INFO: `/v3/projects/1be2495cc1f8452f9b0f95a973e7aceb/users/81333f53d1d440189754ffa4bac3864d/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.671136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82617070-a841-48df-85bb-970f15d9e60f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1be2495cc1f8452f9b0f95a973e7aceb, user_id=81333f53d1d440189754ffa4bac3864d, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.675367 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82617070-a841-48df-85bb-970f15d9e60f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.702140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 903/1796] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:19 2022] PUT /identity/v3/projects/1be2495cc1f8452f9b0f95a973e7aceb/users/81333f53d1d440189754ffa4bac3864d/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:14:19.705102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c580f6-bf0d-4d4d-b03e-299a9fe87341 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.705408 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c580f6-bf0d-4d4d-b03e-299a9fe87341 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.705688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c580f6-bf0d-4d4d-b03e-299a9fe87341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.716390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:19.722978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82617070-a841-48df-85bb-970f15d9e60f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.743256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a1c580f6-bf0d-4d4d-b03e-299a9fe87341 None None] MFA Rules not processed for user `81333f53d1d440189754ffa4bac3864d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:19.744565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a1c580f6-bf0d-4d4d-b03e-299a9fe87341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:19.785834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82617070-a841-48df-85bb-970f15d9e60f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.789775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1078237-04b1-40e5-96c8-eb6b27bbb8d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.790394 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078237-04b1-40e5-96c8-eb6b27bbb8d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.790679 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078237-04b1-40e5-96c8-eb6b27bbb8d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.790949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078237-04b1-40e5-96c8-eb6b27bbb8d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.793071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b1078237-04b1-40e5-96c8-eb6b27bbb8d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:19.844590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 894/1797] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:19.849180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c580f6-bf0d-4d4d-b03e-299a9fe87341 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.853062 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:19.870230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1078237-04b1-40e5-96c8-eb6b27bbb8d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.872497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1078237-04b1-40e5-96c8-eb6b27bbb8d8 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.882296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 904/1798] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:14:19.925817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c580f6-bf0d-4d4d-b03e-299a9fe87341 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:19.931245 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d6cbc-c71a-448d-8991-3fd6eb6ad97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:19.932062 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d6cbc-c71a-448d-8991-3fd6eb6ad97b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:19.932361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d6cbc-c71a-448d-8991-3fd6eb6ad97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:19.932680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d6cbc-c71a-448d-8991-3fd6eb6ad97b admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:19.933466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d6cbc-c71a-448d-8991-3fd6eb6ad97b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:19.935518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d6cbc-c71a-448d-8991-3fd6eb6ad97b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:19.970067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 895/1799] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:14:19 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:14:19.972221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1078237-04b1-40e5-96c8-eb6b27bbb8d8 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:19.983806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1078237-04b1-40e5-96c8-eb6b27bbb8d8 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:20.057473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383eea56-943b-4a84-a87b-60997f4630e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:20.058182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383eea56-943b-4a84-a87b-60997f4630e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:20.058343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383eea56-943b-4a84-a87b-60997f4630e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:20.058605 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383eea56-943b-4a84-a87b-60997f4630e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:20.059753 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383eea56-943b-4a84-a87b-60997f4630e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:20.061412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383eea56-943b-4a84-a87b-60997f4630e3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:20.084461 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 905/1800] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:14:19 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:20.088624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d6cbc-c71a-448d-8991-3fd6eb6ad97b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:20.101752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d6cbc-c71a-448d-8991-3fd6eb6ad97b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:20.106581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68d24b-72fd-404f-bb8e-ba7376515f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:20.107303 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68d24b-72fd-404f-bb8e-ba7376515f53 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:20.107584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68d24b-72fd-404f-bb8e-ba7376515f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:20.107857 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68d24b-72fd-404f-bb8e-ba7376515f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:20.108530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68d24b-72fd-404f-bb8e-ba7376515f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:20.110185 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68d24b-72fd-404f-bb8e-ba7376515f53 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:20.124098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 896/1801] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:20.125586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383eea56-943b-4a84-a87b-60997f4630e3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:20.135920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383eea56-943b-4a84-a87b-60997f4630e3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:20.140580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee3429-6886-436f-9e12-cc948260d819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:20.141220 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee3429-6886-436f-9e12-cc948260d819 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:20.141498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee3429-6886-436f-9e12-cc948260d819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:20.141767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee3429-6886-436f-9e12-cc948260d819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:20.142407 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee3429-6886-436f-9e12-cc948260d819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:20.144144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee3429-6886-436f-9e12-cc948260d819 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:20.154500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 906/1802] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:20.158892 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68d24b-72fd-404f-bb8e-ba7376515f53 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:20.169774 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68d24b-72fd-404f-bb8e-ba7376515f53 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:20.175684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dea5db-380b-469a-88a4-cf7b668fe7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:20.176551 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dea5db-380b-469a-88a4-cf7b668fe7e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:20.176934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dea5db-380b-469a-88a4-cf7b668fe7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:20.177087 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dea5db-380b-469a-88a4-cf7b668fe7e7 admin admin] PATH_INFO: `/v3/projects/5fd254ee7941409c93d04fa56d592c9d/users/ffb0ef5cb3f9482fa71ebaa563d3cb67/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:20.193681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dea5db-380b-469a-88a4-cf7b668fe7e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fd254ee7941409c93d04fa56d592c9d, user_id=ffb0ef5cb3f9482fa71ebaa563d3cb67, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:20.196389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dea5db-380b-469a-88a4-cf7b668fe7e7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:20.211397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 897/1803] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:20 2022] PUT /identity/v3/projects/5fd254ee7941409c93d04fa56d592c9d/users/ffb0ef5cb3f9482fa71ebaa563d3cb67/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:20.223025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa49354-186b-437a-9463-1b09d7279969 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:20.223341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa49354-186b-437a-9463-1b09d7279969 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:20.223608 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa49354-186b-437a-9463-1b09d7279969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:20.251616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0aa49354-186b-437a-9463-1b09d7279969 None None] MFA Rules not processed for user `ffb0ef5cb3f9482fa71ebaa563d3cb67`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:20.252942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0aa49354-186b-437a-9463-1b09d7279969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:20.328834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 907/1804] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:20.333730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dea5db-380b-469a-88a4-cf7b668fe7e7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:20.381550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:20.399482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dea5db-380b-469a-88a4-cf7b668fe7e7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:20.404227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8dc6e-a136-4925-a6f7-08ef041e0919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:20.404871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8dc6e-a136-4925-a6f7-08ef041e0919 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:20.405148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8dc6e-a136-4925-a6f7-08ef041e0919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:20.405418 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8dc6e-a136-4925-a6f7-08ef041e0919 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:20.406047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8dc6e-a136-4925-a6f7-08ef041e0919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:20.407982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8dc6e-a136-4925-a6f7-08ef041e0919 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:20.440908 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 898/1805] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:14:20 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:20.442292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa49354-186b-437a-9463-1b09d7279969 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:20.454023 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa49354-186b-437a-9463-1b09d7279969 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:20.513842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:20.522773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e658a-a91a-497a-8cf8-bc794a2f02e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:20.523410 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e658a-a91a-497a-8cf8-bc794a2f02e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:20.523688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e658a-a91a-497a-8cf8-bc794a2f02e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:20.524005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e658a-a91a-497a-8cf8-bc794a2f02e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:20.524707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e658a-a91a-497a-8cf8-bc794a2f02e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:20.526654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e658a-a91a-497a-8cf8-bc794a2f02e2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:20.551543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 908/1806] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:14:20 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:20.556572 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8dc6e-a136-4925-a6f7-08ef041e0919 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:20.592929 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8dc6e-a136-4925-a6f7-08ef041e0919 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:20.600147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155847e0-086d-4bf5-a055-274cd591b566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:20.600753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155847e0-086d-4bf5-a055-274cd591b566 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:20.601681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155847e0-086d-4bf5-a055-274cd591b566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:20.601914 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155847e0-086d-4bf5-a055-274cd591b566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:20.602597 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155847e0-086d-4bf5-a055-274cd591b566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:20.608362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155847e0-086d-4bf5-a055-274cd591b566 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:20.628132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 899/1807] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:20.629366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e658a-a91a-497a-8cf8-bc794a2f02e2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:20.640274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e658a-a91a-497a-8cf8-bc794a2f02e2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:20.645520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4e042-65fd-4756-86e5-316ab7bae2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:20.646210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4e042-65fd-4756-86e5-316ab7bae2ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:20.646490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4e042-65fd-4756-86e5-316ab7bae2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:20.648889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4e042-65fd-4756-86e5-316ab7bae2ec admin admin] PATH_INFO: `/v3/projects/10d41b36a4f149b38d4d3d1a74f662a0/users/083a4f82397745c69bf7d848af3d12b3/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:20.665937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4e042-65fd-4756-86e5-316ab7bae2ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d41b36a4f149b38d4d3d1a74f662a0, user_id=083a4f82397745c69bf7d848af3d12b3, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:20.668955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4e042-65fd-4756-86e5-316ab7bae2ec admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:20.693552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 909/1808] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:20 2022] PUT /identity/v3/projects/10d41b36a4f149b38d4d3d1a74f662a0/users/083a4f82397745c69bf7d848af3d12b3/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:14:20.700477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155847e0-086d-4bf5-a055-274cd591b566 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:20.712601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155847e0-086d-4bf5-a055-274cd591b566 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:20.802280 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457d57d-d7ed-472c-bb67-b8afd866c869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:20.802944 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457d57d-d7ed-472c-bb67-b8afd866c869 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:20.803222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457d57d-d7ed-472c-bb67-b8afd866c869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:20.803495 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457d57d-d7ed-472c-bb67-b8afd866c869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:20.804139 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457d57d-d7ed-472c-bb67-b8afd866c869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:20.805841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457d57d-d7ed-472c-bb67-b8afd866c869 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:20.820721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 900/1809] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:20.822455 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4e042-65fd-4756-86e5-316ab7bae2ec admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:20.855196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4e042-65fd-4756-86e5-316ab7bae2ec admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:20.861312 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16443832-5292-49ee-9f3b-6681d9b558be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:20.862936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16443832-5292-49ee-9f3b-6681d9b558be admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:20.863478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16443832-5292-49ee-9f3b-6681d9b558be admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:20.864025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16443832-5292-49ee-9f3b-6681d9b558be admin admin] PATH_INFO: `/v3/projects/10d41b36a4f149b38d4d3d1a74f662a0/users/083a4f82397745c69bf7d848af3d12b3/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:20.867213 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16443832-5292-49ee-9f3b-6681d9b558be admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d41b36a4f149b38d4d3d1a74f662a0, user_id=083a4f82397745c69bf7d848af3d12b3, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:20.873730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16443832-5292-49ee-9f3b-6681d9b558be admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:20.887134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 910/1810] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:20 2022] PUT /identity/v3/projects/10d41b36a4f149b38d4d3d1a74f662a0/users/083a4f82397745c69bf7d848af3d12b3/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:14:20.899277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52c331-4754-44d4-a03d-6550f94d1cb6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:20.899562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52c331-4754-44d4-a03d-6550f94d1cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:20.899834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52c331-4754-44d4-a03d-6550f94d1cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:20.938126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8f52c331-4754-44d4-a03d-6550f94d1cb6 None None] MFA Rules not processed for user `083a4f82397745c69bf7d848af3d12b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:20.939562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8f52c331-4754-44d4-a03d-6550f94d1cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:21.020295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 901/1811] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:21.032099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16443832-5292-49ee-9f3b-6681d9b558be admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:21.108358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16443832-5292-49ee-9f3b-6681d9b558be admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:21.115660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455288be-30ff-4e7e-ba26-674d105a28cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:21.115660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455288be-30ff-4e7e-ba26-674d105a28cc service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:21.115660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455288be-30ff-4e7e-ba26-674d105a28cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:21.115660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455288be-30ff-4e7e-ba26-674d105a28cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:21.138771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455288be-30ff-4e7e-ba26-674d105a28cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:21.140627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455288be-30ff-4e7e-ba26-674d105a28cc service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:21.164112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 911/1812] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:21.224431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52c331-4754-44d4-a03d-6550f94d1cb6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:21.235742 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52c331-4754-44d4-a03d-6550f94d1cb6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:21.239666 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465cefa-aaef-4b4e-8519-741b438919b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:21.240288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465cefa-aaef-4b4e-8519-741b438919b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:21.240568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465cefa-aaef-4b4e-8519-741b438919b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:21.242469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465cefa-aaef-4b4e-8519-741b438919b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:21.322640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:21.372402 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465cefa-aaef-4b4e-8519-741b438919b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:21.374538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465cefa-aaef-4b4e-8519-741b438919b4 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:21.388099 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 902/1813] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:21.847598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455288be-30ff-4e7e-ba26-674d105a28cc service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:21.849714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-455288be-30ff-4e7e-ba26-674d105a28cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:21.864599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455288be-30ff-4e7e-ba26-674d105a28cc service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:21.880910 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f6d6d-f172-45e8-b2a3-0b16dd1224d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:21.883252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f6d6d-f172-45e8-b2a3-0b16dd1224d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:21.883859 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f6d6d-f172-45e8-b2a3-0b16dd1224d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:21.884673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f6d6d-f172-45e8-b2a3-0b16dd1224d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:21.887481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c97f6d6d-f172-45e8-b2a3-0b16dd1224d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:22.040144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f6d6d-f172-45e8-b2a3-0b16dd1224d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:22.041560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f6d6d-f172-45e8-b2a3-0b16dd1224d8 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:22.060803 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 912/1814] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:31.382200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=10 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:32.047457 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:50.111575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465cefa-aaef-4b4e-8519-741b438919b4 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:50.184985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465cefa-aaef-4b4e-8519-741b438919b4 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:50.189943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:50.190659 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:50.190972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:50.191346 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] PATH_INFO: `/v3/users/81333f53d1d440189754ffa4bac3864d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:50.192618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=81333f53d1d440189754ffa4bac3864d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:50.194642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:50.255302 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81333f53d1d440189754ffa4bac3864d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:50.255980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81333f53d1d440189754ffa4bac3864d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:50.263431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81333f53d1d440189754ffa4bac3864d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:50.284873 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81333f53d1d440189754ffa4bac3864d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:50.285629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81333f53d1d440189754ffa4bac3864d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:50.306384 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 903/1815] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:14:50 2022] DELETE /identity/v3/users/81333f53d1d440189754ffa4bac3864d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:50.311093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b66ec-20ef-4921-9d7e-206aa742c99c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:50.311557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b66ec-20ef-4921-9d7e-206aa742c99c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:50.311651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b66ec-20ef-4921-9d7e-206aa742c99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:50.332824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-875b66ec-20ef-4921-9d7e-206aa742c99c None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:50.334052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-875b66ec-20ef-4921-9d7e-206aa742c99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:50.403370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 913/1816] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:50.413443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:50.415012 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:50.472816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16b506-5fe0-46fa-8e7a-4aab7313f0ad admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:50.476513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b548a-8114-4f5e-b20a-6991b9421e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:50.477152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b548a-8114-4f5e-b20a-6991b9421e79 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:50.477427 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b548a-8114-4f5e-b20a-6991b9421e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:50.477696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b548a-8114-4f5e-b20a-6991b9421e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:50.487522 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b548a-8114-4f5e-b20a-6991b9421e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:50.489120 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b548a-8114-4f5e-b20a-6991b9421e79 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:50.499909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 904/1817] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:50.807942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b66ec-20ef-4921-9d7e-206aa742c99c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:50.810160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-875b66ec-20ef-4921-9d7e-206aa742c99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:50.828674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b66ec-20ef-4921-9d7e-206aa742c99c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:50.841969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:50.842712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:50.843057 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:50.843552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] PATH_INFO: `/v3/projects/1be2495cc1f8452f9b0f95a973e7aceb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:50.845592 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1be2495cc1f8452f9b0f95a973e7aceb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:50.848839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:50.947309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1be2495cc1f8452f9b0f95a973e7aceb', 'request_id': 'req-74dd7007-151b-4d38-8a77-67290d8d1f83'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:50.960692 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1be2495cc1f8452f9b0f95a973e7aceb', 'request_id': 'req-74dd7007-151b-4d38-8a77-67290d8d1f83'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:50.973817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1be2495cc1f8452f9b0f95a973e7aceb', 'request_id': 'req-74dd7007-151b-4d38-8a77-67290d8d1f83'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:50.996084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] The token cache is being invalidate because project 1be2495cc1f8452f9b0f95a973e7aceb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:14:50.996633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:50.997466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:50.999434 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 914/1818] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:14:50 2022] DELETE /identity/v3/projects/1be2495cc1f8452f9b0f95a973e7aceb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:51.003865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b548a-8114-4f5e-b20a-6991b9421e79 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:51.015724 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b548a-8114-4f5e-b20a-6991b9421e79 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:51.047174 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:51.097824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:51.098657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:51.098987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:51.099276 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] PATH_INFO: `/v3/users/ffb0ef5cb3f9482fa71ebaa563d3cb67` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:51.100489 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb0ef5cb3f9482fa71ebaa563d3cb67)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:51.102911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:51.174014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb0ef5cb3f9482fa71ebaa563d3cb67'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:51.175121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb0ef5cb3f9482fa71ebaa563d3cb67'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:51.183950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb0ef5cb3f9482fa71ebaa563d3cb67'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:51.208034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb0ef5cb3f9482fa71ebaa563d3cb67'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:51.208883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb0ef5cb3f9482fa71ebaa563d3cb67'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:51.233260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 905/1819] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:14:51 2022] DELETE /identity/v3/users/ffb0ef5cb3f9482fa71ebaa563d3cb67 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:14:51.704106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:51.705973 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:51.722698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd7007-151b-4d38-8a77-67290d8d1f83 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:51.786514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:51.786842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:51.786972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:51.787333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] PATH_INFO: `/v3/projects/5fd254ee7941409c93d04fa56d592c9d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:51.788331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fd254ee7941409c93d04fa56d592c9d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:51.790331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:51.825851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:51.849852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fd254ee7941409c93d04fa56d592c9d', 'request_id': 'req-66b9bb4b-1278-4e91-8676-08c7ed7fa007'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:51.856306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fd254ee7941409c93d04fa56d592c9d', 'request_id': 'req-66b9bb4b-1278-4e91-8676-08c7ed7fa007'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:51.862107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fd254ee7941409c93d04fa56d592c9d', 'request_id': 'req-66b9bb4b-1278-4e91-8676-08c7ed7fa007'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:51.877078 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] The token cache is being invalidate because project 5fd254ee7941409c93d04fa56d592c9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:14:51.877383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:51.878172 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:51.879892 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 915/1820] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:14:51 2022] DELETE /identity/v3/projects/5fd254ee7941409c93d04fa56d592c9d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:51.884103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:51.885630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:51.899133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7adfcc6-be83-437f-a76b-317b83825dd8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:51.957819 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddb565-5c7a-42c1-a7b8-f915bcf8ec52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:51.958456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddb565-5c7a-42c1-a7b8-f915bcf8ec52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:51.958731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddb565-5c7a-42c1-a7b8-f915bcf8ec52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:51.959004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddb565-5c7a-42c1-a7b8-f915bcf8ec52 admin admin] PATH_INFO: `/v3/users/083a4f82397745c69bf7d848af3d12b3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:51.960134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddb565-5c7a-42c1-a7b8-f915bcf8ec52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=083a4f82397745c69bf7d848af3d12b3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:51.962193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddb565-5c7a-42c1-a7b8-f915bcf8ec52 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:52.010439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:52.013166 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9dddb565-5c7a-42c1-a7b8-f915bcf8ec52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '083a4f82397745c69bf7d848af3d12b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:52.013927 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9dddb565-5c7a-42c1-a7b8-f915bcf8ec52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '083a4f82397745c69bf7d848af3d12b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:52.022713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9dddb565-5c7a-42c1-a7b8-f915bcf8ec52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '083a4f82397745c69bf7d848af3d12b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:52.037504 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9dddb565-5c7a-42c1-a7b8-f915bcf8ec52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '083a4f82397745c69bf7d848af3d12b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:52.038259 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9dddb565-5c7a-42c1-a7b8-f915bcf8ec52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '083a4f82397745c69bf7d848af3d12b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:52.057651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 906/1821] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:14:51 2022] DELETE /identity/v3/users/083a4f82397745c69bf7d848af3d12b3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:52.371421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:52.372887 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:52.392721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9bb4b-1278-4e91-8676-08c7ed7fa007 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:52.458886 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43bacd-520e-4d4a-9069-101ce520387e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:52.459525 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43bacd-520e-4d4a-9069-101ce520387e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:52.459802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43bacd-520e-4d4a-9069-101ce520387e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:52.460069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43bacd-520e-4d4a-9069-101ce520387e admin admin] PATH_INFO: `/v3/projects/10d41b36a4f149b38d4d3d1a74f662a0` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:52.461206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43bacd-520e-4d4a-9069-101ce520387e admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d41b36a4f149b38d4d3d1a74f662a0)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:52.463145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43bacd-520e-4d4a-9069-101ce520387e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:52.510021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a43bacd-520e-4d4a-9069-101ce520387e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d41b36a4f149b38d4d3d1a74f662a0', 'request_id': 'req-5a43bacd-520e-4d4a-9069-101ce520387e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:52.516288 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a43bacd-520e-4d4a-9069-101ce520387e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d41b36a4f149b38d4d3d1a74f662a0', 'request_id': 'req-5a43bacd-520e-4d4a-9069-101ce520387e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:52.522142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a43bacd-520e-4d4a-9069-101ce520387e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d41b36a4f149b38d4d3d1a74f662a0', 'request_id': 'req-5a43bacd-520e-4d4a-9069-101ce520387e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:52.536766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a43bacd-520e-4d4a-9069-101ce520387e admin admin] The token cache is being invalidate because project 10d41b36a4f149b38d4d3d1a74f662a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:14:52.537057 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a43bacd-520e-4d4a-9069-101ce520387e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:52.537702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a43bacd-520e-4d4a-9069-101ce520387e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:52.539182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 916/1822] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:14:52 2022] DELETE /identity/v3/projects/10d41b36a4f149b38d4d3d1a74f662a0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:52.545781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d703652-9634-45f1-a609-882914576329 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:52.547345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d703652-9634-45f1-a609-882914576329 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:52.547345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d703652-9634-45f1-a609-882914576329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:52.570775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4d703652-9634-45f1-a609-882914576329 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:52.572007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4d703652-9634-45f1-a609-882914576329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:52.645191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 907/1823] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:52.653738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c1bda-3ee1-4679-a80d-4f79403f8f09 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:52.654026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c1bda-3ee1-4679-a80d-4f79403f8f09 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:52.654295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c1bda-3ee1-4679-a80d-4f79403f8f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:52.677538 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f93c1bda-3ee1-4679-a80d-4f79403f8f09 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:52.678972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f93c1bda-3ee1-4679-a80d-4f79403f8f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:52.689036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 917/1824] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:52.693986 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d703652-9634-45f1-a609-882914576329 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:52.704983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d703652-9634-45f1-a609-882914576329 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:52.711016 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e4d2e-ce35-4079-a6f7-f8bdbeb199bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:52.711640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e4d2e-ce35-4079-a6f7-f8bdbeb199bf admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:52.712076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e4d2e-ce35-4079-a6f7-f8bdbeb199bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:52.712353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e4d2e-ce35-4079-a6f7-f8bdbeb199bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:52.713049 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e4d2e-ce35-4079-a6f7-f8bdbeb199bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:14:52.713342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e4d2e-ce35-4079-a6f7-f8bdbeb199bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:52.715567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e4d2e-ce35-4079-a6f7-f8bdbeb199bf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:52.733014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 908/1825] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:14:52.737348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c1bda-3ee1-4679-a80d-4f79403f8f09 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:52.749745 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c1bda-3ee1-4679-a80d-4f79403f8f09 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:52.756492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd9da2-1ba5-4b1c-9beb-50cc20d185d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:52.757194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd9da2-1ba5-4b1c-9beb-50cc20d185d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:52.757523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd9da2-1ba5-4b1c-9beb-50cc20d185d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:52.757845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd9da2-1ba5-4b1c-9beb-50cc20d185d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:52.758884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd9da2-1ba5-4b1c-9beb-50cc20d185d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:52.761542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd9da2-1ba5-4b1c-9beb-50cc20d185d3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:52.796851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 918/1826] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:14:52 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:52.802923 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e4d2e-ce35-4079-a6f7-f8bdbeb199bf admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:52.817661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e4d2e-ce35-4079-a6f7-f8bdbeb199bf admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:52.898685 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee35b2-5e0c-4b11-9e1a-eb94a331e447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:52.899535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee35b2-5e0c-4b11-9e1a-eb94a331e447 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:52.899989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee35b2-5e0c-4b11-9e1a-eb94a331e447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:52.900311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee35b2-5e0c-4b11-9e1a-eb94a331e447 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:52.901354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee35b2-5e0c-4b11-9e1a-eb94a331e447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:52.903385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee35b2-5e0c-4b11-9e1a-eb94a331e447 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:52.936450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 909/1827] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:14:52 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:52.940861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd9da2-1ba5-4b1c-9beb-50cc20d185d3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:52.957513 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd9da2-1ba5-4b1c-9beb-50cc20d185d3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:52.963127 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf10c14-d4e6-49c5-8c2e-2d96b2b7dca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:52.964044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf10c14-d4e6-49c5-8c2e-2d96b2b7dca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:52.964432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf10c14-d4e6-49c5-8c2e-2d96b2b7dca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:52.964849 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf10c14-d4e6-49c5-8c2e-2d96b2b7dca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:52.965750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf10c14-d4e6-49c5-8c2e-2d96b2b7dca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:52.967818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf10c14-d4e6-49c5-8c2e-2d96b2b7dca4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:52.979551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 919/1828] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:52.984634 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee35b2-5e0c-4b11-9e1a-eb94a331e447 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:53.001807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee35b2-5e0c-4b11-9e1a-eb94a331e447 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:53.008681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e79fe-8e1c-4dab-b8c1-636758a0736d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:53.009653 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e79fe-8e1c-4dab-b8c1-636758a0736d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:53.010027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e79fe-8e1c-4dab-b8c1-636758a0736d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:53.010339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e79fe-8e1c-4dab-b8c1-636758a0736d admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:53.010985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e79fe-8e1c-4dab-b8c1-636758a0736d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:53.012791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e79fe-8e1c-4dab-b8c1-636758a0736d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:53.026595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 910/1829] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:53.031298 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf10c14-d4e6-49c5-8c2e-2d96b2b7dca4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:53.038451 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:53.043049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf10c14-d4e6-49c5-8c2e-2d96b2b7dca4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:53.055562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee55b76-9c6c-412e-a38d-467cbc612ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:53.056345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee55b76-9c6c-412e-a38d-467cbc612ed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:53.056729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee55b76-9c6c-412e-a38d-467cbc612ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:53.057092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee55b76-9c6c-412e-a38d-467cbc612ed1 admin admin] PATH_INFO: `/v3/projects/b93ff6b02eae473db54c9fa3d5633248/users/4997f3e09ed14a5f89c5815bd8ee7255/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:53.072448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee55b76-9c6c-412e-a38d-467cbc612ed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b93ff6b02eae473db54c9fa3d5633248, user_id=4997f3e09ed14a5f89c5815bd8ee7255, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:53.075101 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee55b76-9c6c-412e-a38d-467cbc612ed1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:53.090222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 920/1830] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:14:53 2022] PUT /identity/v3/projects/b93ff6b02eae473db54c9fa3d5633248/users/4997f3e09ed14a5f89c5815bd8ee7255/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:53.103113 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116998ac-1de8-46b2-b1a9-152c89d115c2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:53.103476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116998ac-1de8-46b2-b1a9-152c89d115c2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:53.103832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116998ac-1de8-46b2-b1a9-152c89d115c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:53.136431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-116998ac-1de8-46b2-b1a9-152c89d115c2 None None] MFA Rules not processed for user `4997f3e09ed14a5f89c5815bd8ee7255`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:53.137640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-116998ac-1de8-46b2-b1a9-152c89d115c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:53.150803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:53.206882 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 911/1831] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:53.218872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee55b76-9c6c-412e-a38d-467cbc612ed1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:53.220684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9ee55b76-9c6c-412e-a38d-467cbc612ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:53.314338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee55b76-9c6c-412e-a38d-467cbc612ed1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:53.318846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1e975-53ae-4d72-a686-2f58610b4e81 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a76937ab52e445c59b9eb9638fd4a753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:53.319491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1e975-53ae-4d72-a686-2f58610b4e81 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:53.319771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1e975-53ae-4d72-a686-2f58610b4e81 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:53.320049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1e975-53ae-4d72-a686-2f58610b4e81 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:53.335532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d1e975-53ae-4d72-a686-2f58610b4e81 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:53.336976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d1e975-53ae-4d72-a686-2f58610b4e81 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:53.347269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 921/1832] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:57.961939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116998ac-1de8-46b2-b1a9-152c89d115c2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:57.963821 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-116998ac-1de8-46b2-b1a9-152c89d115c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:58.045405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116998ac-1de8-46b2-b1a9-152c89d115c2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:58.054992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:58.055749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:58.056099 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:58.056459 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] PATH_INFO: `/v3/users/114eaad788dd479a8cf92a7e0eb18ff2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:58.057713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] RBAC: Authorizing `identity:delete_user(user_id=114eaad788dd479a8cf92a7e0eb18ff2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:58.059736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:58.120962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114eaad788dd479a8cf92a7e0eb18ff2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:58.121710 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114eaad788dd479a8cf92a7e0eb18ff2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:58.129425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114eaad788dd479a8cf92a7e0eb18ff2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:58.144057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114eaad788dd479a8cf92a7e0eb18ff2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:58.144994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114eaad788dd479a8cf92a7e0eb18ff2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:58.158040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 912/1833] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:14:57 2022] DELETE /identity/v3/users/114eaad788dd479a8cf92a7e0eb18ff2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:58.162680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0155a6-b30f-42f2-9e59-78e8c68cb575 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:58.163247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0155a6-b30f-42f2-9e59-78e8c68cb575 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:58.163345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0155a6-b30f-42f2-9e59-78e8c68cb575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:58.184010 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1c0155a6-b30f-42f2-9e59-78e8c68cb575 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:58.185374 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1c0155a6-b30f-42f2-9e59-78e8c68cb575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:58.247253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 922/1834] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:58.257781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:58.259388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:58.307496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:58.332527 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c15b5-5de9-4faf-89c7-e2f5ce7b752f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:58.336830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693e10a-d6d4-4b50-a0bf-0a13efc363e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:58.337441 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693e10a-d6d4-4b50-a0bf-0a13efc363e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:58.337713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693e10a-d6d4-4b50-a0bf-0a13efc363e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:58.337977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693e10a-d6d4-4b50-a0bf-0a13efc363e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:58.347963 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693e10a-d6d4-4b50-a0bf-0a13efc363e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:58.349603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693e10a-d6d4-4b50-a0bf-0a13efc363e5 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:58.361840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 913/1835] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:14:58.682980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0155a6-b30f-42f2-9e59-78e8c68cb575 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:58.685048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1c0155a6-b30f-42f2-9e59-78e8c68cb575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:58.698760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0155a6-b30f-42f2-9e59-78e8c68cb575 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:58.704687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:58.708252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:58.708252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:58.708252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] PATH_INFO: `/v3/projects/5f6d5fe8091b4954be27a235d286089d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:58.710274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f6d5fe8091b4954be27a235d286089d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:58.711372 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:58.777070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f6d5fe8091b4954be27a235d286089d', 'request_id': 'req-c70a96ac-e496-4d72-bf52-c4ec759d4b43'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:58.784750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f6d5fe8091b4954be27a235d286089d', 'request_id': 'req-c70a96ac-e496-4d72-bf52-c4ec759d4b43'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:58.791845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:58.793768 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f6d5fe8091b4954be27a235d286089d', 'request_id': 'req-c70a96ac-e496-4d72-bf52-c4ec759d4b43'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:58.815287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] The token cache is being invalidate because project 5f6d5fe8091b4954be27a235d286089d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:14:58.815676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:58.816504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:58.818077 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 923/1836] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:14:58 2022] DELETE /identity/v3/projects/5f6d5fe8091b4954be27a235d286089d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:58.822128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693e10a-d6d4-4b50-a0bf-0a13efc363e5 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:58.847497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693e10a-d6d4-4b50-a0bf-0a13efc363e5 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:58.928950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3be4e-36ab-498c-942f-10c31d9d1d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:58.929601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3be4e-36ab-498c-942f-10c31d9d1d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:58.929875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3be4e-36ab-498c-942f-10c31d9d1d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:58.930150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3be4e-36ab-498c-942f-10c31d9d1d7c admin admin] PATH_INFO: `/v3/users/f10667a78b2e4aa099efe0fb7e67f387` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:58.931273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3be4e-36ab-498c-942f-10c31d9d1d7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f10667a78b2e4aa099efe0fb7e67f387)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:58.933306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3be4e-36ab-498c-942f-10c31d9d1d7c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:58.986093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8ea3be4e-36ab-498c-942f-10c31d9d1d7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f10667a78b2e4aa099efe0fb7e67f387'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:58.987359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8ea3be4e-36ab-498c-942f-10c31d9d1d7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f10667a78b2e4aa099efe0fb7e67f387'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:58.995473 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8ea3be4e-36ab-498c-942f-10c31d9d1d7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f10667a78b2e4aa099efe0fb7e67f387'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:59.007905 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8ea3be4e-36ab-498c-942f-10c31d9d1d7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f10667a78b2e4aa099efe0fb7e67f387'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:59.008685 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8ea3be4e-36ab-498c-942f-10c31d9d1d7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f10667a78b2e4aa099efe0fb7e67f387'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:59.023686 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 914/1837] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:14:58 2022] DELETE /identity/v3/users/f10667a78b2e4aa099efe0fb7e67f387 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:59.401896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:59.403931 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:59.428212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a96ac-e496-4d72-bf52-c4ec759d4b43 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:59.492482 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b5792-ff1f-40f3-a07b-d414b1015c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:59.493175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b5792-ff1f-40f3-a07b-d414b1015c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:59.493458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b5792-ff1f-40f3-a07b-d414b1015c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:59.495082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b5792-ff1f-40f3-a07b-d414b1015c89 admin admin] PATH_INFO: `/v3/projects/b51a2462858348c482e2423d706abd1b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:59.495082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b5792-ff1f-40f3-a07b-d414b1015c89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b51a2462858348c482e2423d706abd1b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:59.497246 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b5792-ff1f-40f3-a07b-d414b1015c89 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:59.547799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2e1b5792-ff1f-40f3-a07b-d414b1015c89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b51a2462858348c482e2423d706abd1b', 'request_id': 'req-2e1b5792-ff1f-40f3-a07b-d414b1015c89'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:59.554851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2e1b5792-ff1f-40f3-a07b-d414b1015c89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b51a2462858348c482e2423d706abd1b', 'request_id': 'req-2e1b5792-ff1f-40f3-a07b-d414b1015c89'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:59.560772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2e1b5792-ff1f-40f3-a07b-d414b1015c89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b51a2462858348c482e2423d706abd1b', 'request_id': 'req-2e1b5792-ff1f-40f3-a07b-d414b1015c89'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:59.573494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2e1b5792-ff1f-40f3-a07b-d414b1015c89 admin admin] The token cache is being invalidate because project b51a2462858348c482e2423d706abd1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:14:59.573942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2e1b5792-ff1f-40f3-a07b-d414b1015c89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:59.574446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2e1b5792-ff1f-40f3-a07b-d414b1015c89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:14:59.576060 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 924/1838] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:14:59 2022] DELETE /identity/v3/projects/b51a2462858348c482e2423d706abd1b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:14:59.587099 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde076d-7603-4faf-aa1a-bcd31133e213 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:59.587691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde076d-7603-4faf-aa1a-bcd31133e213 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:59.588237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde076d-7603-4faf-aa1a-bcd31133e213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:59.611897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bdde076d-7603-4faf-aa1a-bcd31133e213 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:59.613370 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bdde076d-7603-4faf-aa1a-bcd31133e213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:59.671481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:14:59.687384 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 915/1839] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:59.690839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e1927-29f4-417a-b55c-b4de7dbaf5e4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:59.691125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e1927-29f4-417a-b55c-b4de7dbaf5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:59.691422 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e1927-29f4-417a-b55c-b4de7dbaf5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:59.713375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-723e1927-29f4-417a-b55c-b4de7dbaf5e4 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:14:59.714590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-723e1927-29f4-417a-b55c-b4de7dbaf5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:14:59.722575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 925/1840] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:14:59.726879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde076d-7603-4faf-aa1a-bcd31133e213 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:59.736880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde076d-7603-4faf-aa1a-bcd31133e213 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:59.741856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246e930-dc8a-4bc9-a140-1a515e89c971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:59.742460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246e930-dc8a-4bc9-a140-1a515e89c971 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:59.742733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246e930-dc8a-4bc9-a140-1a515e89c971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:59.743003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246e930-dc8a-4bc9-a140-1a515e89c971 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:59.743657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246e930-dc8a-4bc9-a140-1a515e89c971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:14:59.743953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246e930-dc8a-4bc9-a140-1a515e89c971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:59.745599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246e930-dc8a-4bc9-a140-1a515e89c971 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:59.758024 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 916/1841] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:14:59.762696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e1927-29f4-417a-b55c-b4de7dbaf5e4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:59.774080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e1927-29f4-417a-b55c-b4de7dbaf5e4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:59.779916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4eb1e-74fb-4776-adc2-b1395e701158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:59.780734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4eb1e-74fb-4776-adc2-b1395e701158 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:59.781012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4eb1e-74fb-4776-adc2-b1395e701158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:59.781279 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4eb1e-74fb-4776-adc2-b1395e701158 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:59.781910 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4eb1e-74fb-4776-adc2-b1395e701158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:59.783850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4eb1e-74fb-4776-adc2-b1395e701158 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:59.815462 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 926/1842] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:14:59 2022] POST /identity/v3/projects => generated 361 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:14:59.821477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246e930-dc8a-4bc9-a140-1a515e89c971 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:59.831983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246e930-dc8a-4bc9-a140-1a515e89c971 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:59.899477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a688ea-b532-403c-af7a-a4c8bd8bab65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:59.900089 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a688ea-b532-403c-af7a-a4c8bd8bab65 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:59.900362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a688ea-b532-403c-af7a-a4c8bd8bab65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:59.900629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a688ea-b532-403c-af7a-a4c8bd8bab65 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:59.901304 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a688ea-b532-403c-af7a-a4c8bd8bab65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:59.903356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a688ea-b532-403c-af7a-a4c8bd8bab65 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:59.932104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 917/1843] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:14:59 2022] POST /identity/v3/users => generated 281 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:14:59.936496 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4eb1e-74fb-4776-adc2-b1395e701158 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:59.948819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4eb1e-74fb-4776-adc2-b1395e701158 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:59.954506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d7572-6fc6-47e2-b5b8-064626aa44a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:59.955209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d7572-6fc6-47e2-b5b8-064626aa44a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:59.955573 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d7572-6fc6-47e2-b5b8-064626aa44a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:59.955896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d7572-6fc6-47e2-b5b8-064626aa44a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:59.956645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d7572-6fc6-47e2-b5b8-064626aa44a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:59.958362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d7572-6fc6-47e2-b5b8-064626aa44a5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:14:59.971156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 927/1844] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:14:59.975491 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a688ea-b532-403c-af7a-a4c8bd8bab65 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:14:59.986377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a688ea-b532-403c-af7a-a4c8bd8bab65 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:14:59.992051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840cb04-81b7-4a09-a380-396231813f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:14:59.992706 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840cb04-81b7-4a09-a380-396231813f13 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:14:59.993011 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840cb04-81b7-4a09-a380-396231813f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:14:59.993279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840cb04-81b7-4a09-a380-396231813f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:14:59.993972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840cb04-81b7-4a09-a380-396231813f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:14:59.995585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840cb04-81b7-4a09-a380-396231813f13 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:00.013446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 918/1845] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:00.017788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d7572-6fc6-47e2-b5b8-064626aa44a5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:00.028418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d7572-6fc6-47e2-b5b8-064626aa44a5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:00.033590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c310c-4c72-4246-a1bf-f6a79d05ae70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:00.034390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c310c-4c72-4246-a1bf-f6a79d05ae70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:00.034840 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c310c-4c72-4246-a1bf-f6a79d05ae70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:00.035905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c310c-4c72-4246-a1bf-f6a79d05ae70 admin admin] PATH_INFO: `/v3/projects/c764346795034c91913a241edeb0b901/users/f26b0459f8d4448c958ae0d655fcab05/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:00.056582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c310c-4c72-4246-a1bf-f6a79d05ae70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c764346795034c91913a241edeb0b901, user_id=f26b0459f8d4448c958ae0d655fcab05, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:00.059546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c310c-4c72-4246-a1bf-f6a79d05ae70 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:00.076111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 928/1846] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:00 2022] PUT /identity/v3/projects/c764346795034c91913a241edeb0b901/users/f26b0459f8d4448c958ae0d655fcab05/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:00.088593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc610dff-f6fb-400b-8e21-109fac0a2b1d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:00.088769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc610dff-f6fb-400b-8e21-109fac0a2b1d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:00.089606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc610dff-f6fb-400b-8e21-109fac0a2b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:00.119606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bc610dff-f6fb-400b-8e21-109fac0a2b1d None None] MFA Rules not processed for user `f26b0459f8d4448c958ae0d655fcab05`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:00.120897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bc610dff-f6fb-400b-8e21-109fac0a2b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:00.191342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 919/1847] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:00.196464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c310c-4c72-4246-a1bf-f6a79d05ae70 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:00.214978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:00.267974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c310c-4c72-4246-a1bf-f6a79d05ae70 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:00.273516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e001f-01bb-4761-9975-9436e08e503f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:00.274168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e001f-01bb-4761-9975-9436e08e503f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:00.274439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e001f-01bb-4761-9975-9436e08e503f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:00.274706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e001f-01bb-4761-9975-9436e08e503f admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:00.275360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e001f-01bb-4761-9975-9436e08e503f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:00.278881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e001f-01bb-4761-9975-9436e08e503f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:00.308808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 929/1848] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:00 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:00.313210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc610dff-f6fb-400b-8e21-109fac0a2b1d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:00.324873 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc610dff-f6fb-400b-8e21-109fac0a2b1d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:00.377853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:00.388174 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cdc9a-9ed0-4b8c-a1b9-a2003749d182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:00.388834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cdc9a-9ed0-4b8c-a1b9-a2003749d182 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:00.389114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cdc9a-9ed0-4b8c-a1b9-a2003749d182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:00.389384 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cdc9a-9ed0-4b8c-a1b9-a2003749d182 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:00.390023 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cdc9a-9ed0-4b8c-a1b9-a2003749d182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:00.392019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cdc9a-9ed0-4b8c-a1b9-a2003749d182 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:00.415197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 920/1849] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:00 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:15:00.419528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e001f-01bb-4761-9975-9436e08e503f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:00.431317 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e001f-01bb-4761-9975-9436e08e503f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:00.436881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e346ec-6b9d-4b3d-9641-584eaf684f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:00.437495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e346ec-6b9d-4b3d-9641-584eaf684f59 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:00.437789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e346ec-6b9d-4b3d-9641-584eaf684f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:00.438059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e346ec-6b9d-4b3d-9641-584eaf684f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:00.438708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e346ec-6b9d-4b3d-9641-584eaf684f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:00.440318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e346ec-6b9d-4b3d-9641-584eaf684f59 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:00.451492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 930/1850] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:00.455841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cdc9a-9ed0-4b8c-a1b9-a2003749d182 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:00.466975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cdc9a-9ed0-4b8c-a1b9-a2003749d182 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:00.471809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828d18d-e3da-4f97-bb16-be3491b08dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:00.472497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828d18d-e3da-4f97-bb16-be3491b08dea admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:00.472825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828d18d-e3da-4f97-bb16-be3491b08dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:00.473086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828d18d-e3da-4f97-bb16-be3491b08dea admin admin] PATH_INFO: `/v3/projects/0a511fa1c3a14cc2b6fba295c035eebc/users/8ce3fbd5740b44dba34a1493905e4a6f/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:00.489934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828d18d-e3da-4f97-bb16-be3491b08dea admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a511fa1c3a14cc2b6fba295c035eebc, user_id=8ce3fbd5740b44dba34a1493905e4a6f, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:00.492594 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828d18d-e3da-4f97-bb16-be3491b08dea admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:00.512112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 921/1851] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:00 2022] PUT /identity/v3/projects/0a511fa1c3a14cc2b6fba295c035eebc/users/8ce3fbd5740b44dba34a1493905e4a6f/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:00.517947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e346ec-6b9d-4b3d-9641-584eaf684f59 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:00.528578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e346ec-6b9d-4b3d-9641-584eaf684f59 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:00.598955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0ee7f-ac7f-4be7-9298-44fb8f80b66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:00.598955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0ee7f-ac7f-4be7-9298-44fb8f80b66d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:00.599952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0ee7f-ac7f-4be7-9298-44fb8f80b66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:00.599952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0ee7f-ac7f-4be7-9298-44fb8f80b66d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:00.600302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0ee7f-ac7f-4be7-9298-44fb8f80b66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:00.602008 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0ee7f-ac7f-4be7-9298-44fb8f80b66d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:00.616557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 931/1852] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:15:00.617970 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828d18d-e3da-4f97-bb16-be3491b08dea admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:00.629438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828d18d-e3da-4f97-bb16-be3491b08dea admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:00.635021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9b229-f4b2-49f3-a6d1-c6f9ea7ef4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:00.635734 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9b229-f4b2-49f3-a6d1-c6f9ea7ef4bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:00.636017 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9b229-f4b2-49f3-a6d1-c6f9ea7ef4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:00.636292 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9b229-f4b2-49f3-a6d1-c6f9ea7ef4bc admin admin] PATH_INFO: `/v3/projects/0a511fa1c3a14cc2b6fba295c035eebc/users/8ce3fbd5740b44dba34a1493905e4a6f/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:00.642659 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9b229-f4b2-49f3-a6d1-c6f9ea7ef4bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a511fa1c3a14cc2b6fba295c035eebc, user_id=8ce3fbd5740b44dba34a1493905e4a6f, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:00.650130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9b229-f4b2-49f3-a6d1-c6f9ea7ef4bc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:00.667446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 922/1853] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:00 2022] PUT /identity/v3/projects/0a511fa1c3a14cc2b6fba295c035eebc/users/8ce3fbd5740b44dba34a1493905e4a6f/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:00.681500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242bfd-66fe-4bb5-9c77-58430f6a5fa9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:00.681778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242bfd-66fe-4bb5-9c77-58430f6a5fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:00.682123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242bfd-66fe-4bb5-9c77-58430f6a5fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:00.714651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-88242bfd-66fe-4bb5-9c77-58430f6a5fa9 None None] MFA Rules not processed for user `8ce3fbd5740b44dba34a1493905e4a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:00.716111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-88242bfd-66fe-4bb5-9c77-58430f6a5fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:00.791964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 932/1854] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:00.805914 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9b229-f4b2-49f3-a6d1-c6f9ea7ef4bc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:00.808480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-91b9b229-f4b2-49f3-a6d1-c6f9ea7ef4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:00.877073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9b229-f4b2-49f3-a6d1-c6f9ea7ef4bc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:00.881099 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8035ac0-29ab-4912-bb5a-40b2c3600187 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:00.881720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8035ac0-29ab-4912-bb5a-40b2c3600187 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:00.881994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8035ac0-29ab-4912-bb5a-40b2c3600187 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:00.882248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8035ac0-29ab-4912-bb5a-40b2c3600187 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:00.892155 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8035ac0-29ab-4912-bb5a-40b2c3600187 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:00.893629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8035ac0-29ab-4912-bb5a-40b2c3600187 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:00.905558 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 923/1855] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:00.989394 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242bfd-66fe-4bb5-9c77-58430f6a5fa9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:00.999667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242bfd-66fe-4bb5-9c77-58430f6a5fa9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:01.003645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488ddc0-17a9-4ddc-b392-8c76b793af77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:01.004186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488ddc0-17a9-4ddc-b392-8c76b793af77 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:01.004461 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488ddc0-17a9-4ddc-b392-8c76b793af77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:01.004764 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488ddc0-17a9-4ddc-b392-8c76b793af77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:01.009997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:01.065054 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488ddc0-17a9-4ddc-b392-8c76b793af77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:01.066566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488ddc0-17a9-4ddc-b392-8c76b793af77 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:01.075524 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 933/1856] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:06.025361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8035ac0-29ab-4912-bb5a-40b2c3600187 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:06.117899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8035ac0-29ab-4912-bb5a-40b2c3600187 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:06.123284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:06.123858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:06.124213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:06.124484 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] PATH_INFO: `/v3/users/f26b0459f8d4448c958ae0d655fcab05` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:06.125679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f26b0459f8d4448c958ae0d655fcab05)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:06.127786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:06.186778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f26b0459f8d4448c958ae0d655fcab05'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:06.187568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f26b0459f8d4448c958ae0d655fcab05'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:06.190306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:06.196555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f26b0459f8d4448c958ae0d655fcab05'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:06.214025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f26b0459f8d4448c958ae0d655fcab05'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:06.214897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f26b0459f8d4448c958ae0d655fcab05'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:06.230119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 924/1857] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:06 2022] DELETE /identity/v3/users/f26b0459f8d4448c958ae0d655fcab05 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:06.234707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d5cb9-8072-4334-b061-75741fb1e545 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:06.235220 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d5cb9-8072-4334-b061-75741fb1e545 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:06.235329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d5cb9-8072-4334-b061-75741fb1e545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:06.256621 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9f8d5cb9-8072-4334-b061-75741fb1e545 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:06.257885 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9f8d5cb9-8072-4334-b061-75741fb1e545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:06.329483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 934/1858] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:06.338116 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:06.339748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:06.429169 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16125704-0a2f-4a39-8c6e-06b96533bc08 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:06.433543 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc8a33-4448-4137-a643-b47a781c6c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:06.434371 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc8a33-4448-4137-a643-b47a781c6c8c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:06.434917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc8a33-4448-4137-a643-b47a781c6c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:06.435383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc8a33-4448-4137-a643-b47a781c6c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:06.450152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc8a33-4448-4137-a643-b47a781c6c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:06.452192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc8a33-4448-4137-a643-b47a781c6c8c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:06.464097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 925/1859] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:06.897292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d5cb9-8072-4334-b061-75741fb1e545 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:06.898870 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9f8d5cb9-8072-4334-b061-75741fb1e545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:06.912644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d5cb9-8072-4334-b061-75741fb1e545 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:06.917784 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:06.919120 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:06.919581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:06.919848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] PATH_INFO: `/v3/projects/c764346795034c91913a241edeb0b901` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:06.921400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c764346795034c91913a241edeb0b901)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:06.923842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:06.983807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c764346795034c91913a241edeb0b901', 'request_id': 'req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:06.994098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c764346795034c91913a241edeb0b901', 'request_id': 'req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.003671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c764346795034c91913a241edeb0b901', 'request_id': 'req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.027756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] The token cache is being invalidate because project c764346795034c91913a241edeb0b901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:07.028500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.029355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.034229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 935/1860] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:06 2022] DELETE /identity/v3/projects/c764346795034c91913a241edeb0b901 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:07.038672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc8a33-4448-4137-a643-b47a781c6c8c service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:07.049681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc8a33-4448-4137-a643-b47a781c6c8c service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:07.122968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a2499-e247-4242-a97c-70e7dc831d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:07.123620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a2499-e247-4242-a97c-70e7dc831d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:07.123891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a2499-e247-4242-a97c-70e7dc831d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:07.124161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a2499-e247-4242-a97c-70e7dc831d6d admin admin] PATH_INFO: `/v3/users/8ce3fbd5740b44dba34a1493905e4a6f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:07.125363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a2499-e247-4242-a97c-70e7dc831d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ce3fbd5740b44dba34a1493905e4a6f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:07.127336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a2499-e247-4242-a97c-70e7dc831d6d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:07.196749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-479a2499-e247-4242-a97c-70e7dc831d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ce3fbd5740b44dba34a1493905e4a6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.200023 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-479a2499-e247-4242-a97c-70e7dc831d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ce3fbd5740b44dba34a1493905e4a6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.214897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-479a2499-e247-4242-a97c-70e7dc831d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ce3fbd5740b44dba34a1493905e4a6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.244558 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-479a2499-e247-4242-a97c-70e7dc831d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ce3fbd5740b44dba34a1493905e4a6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.245562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:07.246832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-479a2499-e247-4242-a97c-70e7dc831d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ce3fbd5740b44dba34a1493905e4a6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.279877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 926/1861] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:07 2022] DELETE /identity/v3/users/8ce3fbd5740b44dba34a1493905e4a6f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:07.680232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:07.689975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:07.707246 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fb042-cbd4-4f73-8bda-298f85e0c2d7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:07.743623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:07.782335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3f8eb-f644-42e8-bebc-64296014a749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:07.782987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3f8eb-f644-42e8-bebc-64296014a749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:07.783261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3f8eb-f644-42e8-bebc-64296014a749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:07.783536 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3f8eb-f644-42e8-bebc-64296014a749 admin admin] PATH_INFO: `/v3/projects/0a511fa1c3a14cc2b6fba295c035eebc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:07.784948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3f8eb-f644-42e8-bebc-64296014a749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a511fa1c3a14cc2b6fba295c035eebc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:07.786800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3f8eb-f644-42e8-bebc-64296014a749 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:07.872005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-50f3f8eb-f644-42e8-bebc-64296014a749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a511fa1c3a14cc2b6fba295c035eebc', 'request_id': 'req-50f3f8eb-f644-42e8-bebc-64296014a749'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.878761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-50f3f8eb-f644-42e8-bebc-64296014a749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a511fa1c3a14cc2b6fba295c035eebc', 'request_id': 'req-50f3f8eb-f644-42e8-bebc-64296014a749'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.885634 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-50f3f8eb-f644-42e8-bebc-64296014a749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a511fa1c3a14cc2b6fba295c035eebc', 'request_id': 'req-50f3f8eb-f644-42e8-bebc-64296014a749'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.900146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-50f3f8eb-f644-42e8-bebc-64296014a749 admin admin] The token cache is being invalidate because project 0a511fa1c3a14cc2b6fba295c035eebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:07.900507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-50f3f8eb-f644-42e8-bebc-64296014a749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.901291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-50f3f8eb-f644-42e8-bebc-64296014a749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:07.903099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 936/1862] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:07 2022] DELETE /identity/v3/projects/0a511fa1c3a14cc2b6fba295c035eebc => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:07.909899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825856d-edfb-49e4-99a6-e3463aa642f6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:07.910447 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825856d-edfb-49e4-99a6-e3463aa642f6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:07.910916 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825856d-edfb-49e4-99a6-e3463aa642f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:07.932355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7825856d-edfb-49e4-99a6-e3463aa642f6 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:07.933778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7825856d-edfb-49e4-99a6-e3463aa642f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:08.002253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 927/1863] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:08.010510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d21ce6-75a7-408e-a480-c2eab002d51c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:08.010796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d21ce6-75a7-408e-a480-c2eab002d51c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:08.011071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d21ce6-75a7-408e-a480-c2eab002d51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:08.031624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d7d21ce6-75a7-408e-a480-c2eab002d51c None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:08.032887 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d7d21ce6-75a7-408e-a480-c2eab002d51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:08.041096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 937/1864] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:08.047082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825856d-edfb-49e4-99a6-e3463aa642f6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:08.058005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825856d-edfb-49e4-99a6-e3463aa642f6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:08.063067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53943707-1ea9-4f8e-8878-2d4daf7df651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:08.063689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53943707-1ea9-4f8e-8878-2d4daf7df651 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:08.063968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53943707-1ea9-4f8e-8878-2d4daf7df651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:08.064242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53943707-1ea9-4f8e-8878-2d4daf7df651 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:08.064933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53943707-1ea9-4f8e-8878-2d4daf7df651 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:15:08.065235 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53943707-1ea9-4f8e-8878-2d4daf7df651 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:08.066933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53943707-1ea9-4f8e-8878-2d4daf7df651 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:08.080350 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 928/1865] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:15:08.085316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d21ce6-75a7-408e-a480-c2eab002d51c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:08.098322 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d21ce6-75a7-408e-a480-c2eab002d51c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:08.103833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae461cbe-5ef6-44f3-b37e-314ef5524325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:08.104443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae461cbe-5ef6-44f3-b37e-314ef5524325 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:08.104745 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae461cbe-5ef6-44f3-b37e-314ef5524325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:08.105017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae461cbe-5ef6-44f3-b37e-314ef5524325 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:08.105659 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae461cbe-5ef6-44f3-b37e-314ef5524325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:08.107622 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae461cbe-5ef6-44f3-b37e-314ef5524325 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:08.136992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 938/1866] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:08 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:08.142026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53943707-1ea9-4f8e-8878-2d4daf7df651 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:08.153081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53943707-1ea9-4f8e-8878-2d4daf7df651 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:08.216347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82eb50-e2ac-48c8-8806-9cfcdb279c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:08.216900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82eb50-e2ac-48c8-8806-9cfcdb279c3e admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:08.217182 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82eb50-e2ac-48c8-8806-9cfcdb279c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:08.217451 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82eb50-e2ac-48c8-8806-9cfcdb279c3e admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:08.218091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c82eb50-e2ac-48c8-8806-9cfcdb279c3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:08.220029 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c82eb50-e2ac-48c8-8806-9cfcdb279c3e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:08.243386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 929/1867] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:08 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:08.249042 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae461cbe-5ef6-44f3-b37e-314ef5524325 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:08.260457 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae461cbe-5ef6-44f3-b37e-314ef5524325 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:08.265690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa8922-5189-4e5c-a14e-b313b1db0e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:08.266315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa8922-5189-4e5c-a14e-b313b1db0e8d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:08.266590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa8922-5189-4e5c-a14e-b313b1db0e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:08.266858 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa8922-5189-4e5c-a14e-b313b1db0e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:08.267514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa8922-5189-4e5c-a14e-b313b1db0e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:08.269242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa8922-5189-4e5c-a14e-b313b1db0e8d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:08.281350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 939/1868] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:15:08.287224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82eb50-e2ac-48c8-8806-9cfcdb279c3e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:08.297847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82eb50-e2ac-48c8-8806-9cfcdb279c3e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:08.303148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfda73c-df79-494c-bf7c-026b42d6772d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:08.303755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfda73c-df79-494c-bf7c-026b42d6772d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:08.304032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfda73c-df79-494c-bf7c-026b42d6772d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:08.304300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfda73c-df79-494c-bf7c-026b42d6772d admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:08.304987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfda73c-df79-494c-bf7c-026b42d6772d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:08.306604 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfda73c-df79-494c-bf7c-026b42d6772d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:08.321702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 930/1869] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:15:08.326175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa8922-5189-4e5c-a14e-b313b1db0e8d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:08.336405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa8922-5189-4e5c-a14e-b313b1db0e8d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:08.341669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df881a91-3b83-4307-a9f7-fb671bb5e727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:08.342330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df881a91-3b83-4307-a9f7-fb671bb5e727 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:08.342606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df881a91-3b83-4307-a9f7-fb671bb5e727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:08.342878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df881a91-3b83-4307-a9f7-fb671bb5e727 admin admin] PATH_INFO: `/v3/projects/075173d6936d416b8e42143278aad225/users/b1d841731b1a4be8aadba142959b6672/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:08.360180 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df881a91-3b83-4307-a9f7-fb671bb5e727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=075173d6936d416b8e42143278aad225, user_id=b1d841731b1a4be8aadba142959b6672, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:08.362959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df881a91-3b83-4307-a9f7-fb671bb5e727 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:08.380009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 940/1870] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:08 2022] PUT /identity/v3/projects/075173d6936d416b8e42143278aad225/users/b1d841731b1a4be8aadba142959b6672/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:08.397736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438f392-a233-4736-91ee-f084b67f0c5f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:08.398031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438f392-a233-4736-91ee-f084b67f0c5f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:08.398336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438f392-a233-4736-91ee-f084b67f0c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:08.439157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c438f392-a233-4736-91ee-f084b67f0c5f None None] MFA Rules not processed for user `b1d841731b1a4be8aadba142959b6672`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:08.440409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c438f392-a233-4736-91ee-f084b67f0c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:08.507385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:08.540348 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 931/1871] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:08.545148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df881a91-3b83-4307-a9f7-fb671bb5e727 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:08.623469 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:08.628184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df881a91-3b83-4307-a9f7-fb671bb5e727 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:08.634627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d301a6-dc7e-45d9-9064-56618ca50bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:08.635248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d301a6-dc7e-45d9-9064-56618ca50bba admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:08.635559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d301a6-dc7e-45d9-9064-56618ca50bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:08.636090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d301a6-dc7e-45d9-9064-56618ca50bba admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:08.636980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d301a6-dc7e-45d9-9064-56618ca50bba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:08.641869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d301a6-dc7e-45d9-9064-56618ca50bba admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:08.673554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 941/1872] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:08 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:08.678264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438f392-a233-4736-91ee-f084b67f0c5f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:08.689382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438f392-a233-4736-91ee-f084b67f0c5f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:08.761125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2396b0d-0302-4aa8-bd79-321ab437a30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:08.762679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2396b0d-0302-4aa8-bd79-321ab437a30c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:08.762679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2396b0d-0302-4aa8-bd79-321ab437a30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:08.762679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2396b0d-0302-4aa8-bd79-321ab437a30c admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:08.763015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2396b0d-0302-4aa8-bd79-321ab437a30c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:08.765651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2396b0d-0302-4aa8-bd79-321ab437a30c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:08.790146 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 932/1873] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:08 2022] POST /identity/v3/users => generated 301 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:08.794416 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d301a6-dc7e-45d9-9064-56618ca50bba admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:08.806062 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d301a6-dc7e-45d9-9064-56618ca50bba admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:08.810815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1789d1f-e2d9-4ad9-9940-10fb9028e818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:08.811427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1789d1f-e2d9-4ad9-9940-10fb9028e818 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:08.811707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1789d1f-e2d9-4ad9-9940-10fb9028e818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:08.811980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1789d1f-e2d9-4ad9-9940-10fb9028e818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:08.812623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1789d1f-e2d9-4ad9-9940-10fb9028e818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:08.814310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1789d1f-e2d9-4ad9-9940-10fb9028e818 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:08.825195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 942/1874] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:08.829779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2396b0d-0302-4aa8-bd79-321ab437a30c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:08.841126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2396b0d-0302-4aa8-bd79-321ab437a30c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:08.847058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eec81-c7bf-48ef-aa30-f1987e431b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:08.847829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eec81-c7bf-48ef-aa30-f1987e431b78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:08.848280 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eec81-c7bf-48ef-aa30-f1987e431b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:08.848805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eec81-c7bf-48ef-aa30-f1987e431b78 admin admin] PATH_INFO: `/v3/projects/8c47248eb06547379f35ff1ff9326f79/users/befd8e33329942108900cfe77c6ea6fe/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:08.865958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0eec81-c7bf-48ef-aa30-f1987e431b78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c47248eb06547379f35ff1ff9326f79, user_id=befd8e33329942108900cfe77c6ea6fe, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:08.868686 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0eec81-c7bf-48ef-aa30-f1987e431b78 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:08.892051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 933/1875] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:08 2022] PUT /identity/v3/projects/8c47248eb06547379f35ff1ff9326f79/users/befd8e33329942108900cfe77c6ea6fe/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:08.896790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1789d1f-e2d9-4ad9-9940-10fb9028e818 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:08.910690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1789d1f-e2d9-4ad9-9940-10fb9028e818 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:08.969598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810822b-675f-464b-b205-5ba31201bc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:08.970226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810822b-675f-464b-b205-5ba31201bc92 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:08.970501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810822b-675f-464b-b205-5ba31201bc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:08.970767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810822b-675f-464b-b205-5ba31201bc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:08.971423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810822b-675f-464b-b205-5ba31201bc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:08.973194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810822b-675f-464b-b205-5ba31201bc92 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:08.984648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 943/1876] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:15:08.989891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eec81-c7bf-48ef-aa30-f1987e431b78 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:09.001110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eec81-c7bf-48ef-aa30-f1987e431b78 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:09.006328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8395a-8e29-4c13-8bc9-a91e28f43aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:09.007070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8395a-8e29-4c13-8bc9-a91e28f43aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:09.007348 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8395a-8e29-4c13-8bc9-a91e28f43aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:09.007623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8395a-8e29-4c13-8bc9-a91e28f43aeb admin admin] PATH_INFO: `/v3/projects/8c47248eb06547379f35ff1ff9326f79/users/befd8e33329942108900cfe77c6ea6fe/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:09.009630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8395a-8e29-4c13-8bc9-a91e28f43aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c47248eb06547379f35ff1ff9326f79, user_id=befd8e33329942108900cfe77c6ea6fe, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:09.012431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8395a-8e29-4c13-8bc9-a91e28f43aeb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:09.027387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 934/1877] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:08 2022] PUT /identity/v3/projects/8c47248eb06547379f35ff1ff9326f79/users/befd8e33329942108900cfe77c6ea6fe/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:09.037978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cee11-3a72-4a2f-9440-02937d41dcd8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:09.038369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cee11-3a72-4a2f-9440-02937d41dcd8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:09.038742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cee11-3a72-4a2f-9440-02937d41dcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:09.108162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-105cee11-3a72-4a2f-9440-02937d41dcd8 None None] MFA Rules not processed for user `befd8e33329942108900cfe77c6ea6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:09.109608 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-105cee11-3a72-4a2f-9440-02937d41dcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:09.182331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 944/1878] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:09.193667 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8395a-8e29-4c13-8bc9-a91e28f43aeb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:09.195598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-49e8395a-8e29-4c13-8bc9-a91e28f43aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:09.267680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8395a-8e29-4c13-8bc9-a91e28f43aeb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:09.272070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fea576-2702-4e2a-a2be-772429736df6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:09.272693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fea576-2702-4e2a-a2be-772429736df6 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:09.272983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fea576-2702-4e2a-a2be-772429736df6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:09.273245 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fea576-2702-4e2a-a2be-772429736df6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:09.284026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fea576-2702-4e2a-a2be-772429736df6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:09.285555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fea576-2702-4e2a-a2be-772429736df6 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:09.295604 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 935/1879] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:09 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:09.439748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105cee11-3a72-4a2f-9440-02937d41dcd8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:09.508519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:09.522638 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105cee11-3a72-4a2f-9440-02937d41dcd8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:09.527752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:09.528394 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:09.528683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:09.528967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] PATH_INFO: `/v3/users/b1d841731b1a4be8aadba142959b6672` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:09.530130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d841731b1a4be8aadba142959b6672)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:09.532063 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:09.581671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d841731b1a4be8aadba142959b6672'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:09.582332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d841731b1a4be8aadba142959b6672'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:09.590934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d841731b1a4be8aadba142959b6672'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:09.604825 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d841731b1a4be8aadba142959b6672'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:09.605598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d841731b1a4be8aadba142959b6672'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:09.620283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 945/1880] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:09 2022] DELETE /identity/v3/users/b1d841731b1a4be8aadba142959b6672 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:09.625175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f55ce2-07d6-4d9c-8d8b-63ae1cbe0a4a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:09.625422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f55ce2-07d6-4d9c-8d8b-63ae1cbe0a4a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:09.625791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f55ce2-07d6-4d9c-8d8b-63ae1cbe0a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:09.648901 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d7f55ce2-07d6-4d9c-8d8b-63ae1cbe0a4a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:09.650236 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d7f55ce2-07d6-4d9c-8d8b-63ae1cbe0a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:09.694656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:09.737935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 936/1881] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:09.750874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:09.752607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:09.830638 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3baac1-36aa-46db-81ca-734c3daf359d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:09.834473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb63fd-8fcc-4af4-a7a4-2faae35f1667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:09.835118 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb63fd-8fcc-4af4-a7a4-2faae35f1667 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:09.835388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb63fd-8fcc-4af4-a7a4-2faae35f1667 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:09.835651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb63fd-8fcc-4af4-a7a4-2faae35f1667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:09.848163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb63fd-8fcc-4af4-a7a4-2faae35f1667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:09.849660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb63fd-8fcc-4af4-a7a4-2faae35f1667 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:09.860080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 946/1882] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:10.159796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f55ce2-07d6-4d9c-8d8b-63ae1cbe0a4a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:10.161456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d7f55ce2-07d6-4d9c-8d8b-63ae1cbe0a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:10.174055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f55ce2-07d6-4d9c-8d8b-63ae1cbe0a4a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:10.179801 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:10.180571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:10.180942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:10.181275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] PATH_INFO: `/v3/projects/075173d6936d416b8e42143278aad225` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:10.182579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=075173d6936d416b8e42143278aad225)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:10.185420 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:10.247288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '075173d6936d416b8e42143278aad225', 'request_id': 'req-da1a65fa-41ec-4f2b-b1ce-292df6886b39'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:10.257698 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '075173d6936d416b8e42143278aad225', 'request_id': 'req-da1a65fa-41ec-4f2b-b1ce-292df6886b39'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:10.276144 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '075173d6936d416b8e42143278aad225', 'request_id': 'req-da1a65fa-41ec-4f2b-b1ce-292df6886b39'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:10.289938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] The token cache is being invalidate because project 075173d6936d416b8e42143278aad225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:10.290237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:10.290921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:10.292592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 937/1883] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:10 2022] DELETE /identity/v3/projects/075173d6936d416b8e42143278aad225 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:10.299158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb63fd-8fcc-4af4-a7a4-2faae35f1667 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:10.313265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb63fd-8fcc-4af4-a7a4-2faae35f1667 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:10.382188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd279cc3-e2b1-4f8f-8e05-708d791f0300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:10.383384 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd279cc3-e2b1-4f8f-8e05-708d791f0300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:10.383857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd279cc3-e2b1-4f8f-8e05-708d791f0300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:10.384358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd279cc3-e2b1-4f8f-8e05-708d791f0300 admin admin] PATH_INFO: `/v3/users/befd8e33329942108900cfe77c6ea6fe` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:10.385923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd279cc3-e2b1-4f8f-8e05-708d791f0300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=befd8e33329942108900cfe77c6ea6fe)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:10.387883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd279cc3-e2b1-4f8f-8e05-708d791f0300 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:10.407620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:10.449051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bd279cc3-e2b1-4f8f-8e05-708d791f0300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'befd8e33329942108900cfe77c6ea6fe'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:10.449724 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bd279cc3-e2b1-4f8f-8e05-708d791f0300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'befd8e33329942108900cfe77c6ea6fe'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:10.458326 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bd279cc3-e2b1-4f8f-8e05-708d791f0300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'befd8e33329942108900cfe77c6ea6fe'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:10.470489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bd279cc3-e2b1-4f8f-8e05-708d791f0300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'befd8e33329942108900cfe77c6ea6fe'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:10.471721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bd279cc3-e2b1-4f8f-8e05-708d791f0300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'befd8e33329942108900cfe77c6ea6fe'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:10.485817 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 947/1884] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:10 2022] DELETE /identity/v3/users/befd8e33329942108900cfe77c6ea6fe => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:10.776333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:10.778335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:10.793294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a65fa-41ec-4f2b-b1ce-292df6886b39 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:10.906664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da68a7-fd71-42c6-bc36-33b6dd49f593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:10.907388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da68a7-fd71-42c6-bc36-33b6dd49f593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:10.908716 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da68a7-fd71-42c6-bc36-33b6dd49f593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:10.908716 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da68a7-fd71-42c6-bc36-33b6dd49f593 admin admin] PATH_INFO: `/v3/projects/8c47248eb06547379f35ff1ff9326f79` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:10.909236 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da68a7-fd71-42c6-bc36-33b6dd49f593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c47248eb06547379f35ff1ff9326f79)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:10.911282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da68a7-fd71-42c6-bc36-33b6dd49f593 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:10.969940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-39da68a7-fd71-42c6-bc36-33b6dd49f593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c47248eb06547379f35ff1ff9326f79', 'request_id': 'req-39da68a7-fd71-42c6-bc36-33b6dd49f593'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:10.982692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-39da68a7-fd71-42c6-bc36-33b6dd49f593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c47248eb06547379f35ff1ff9326f79', 'request_id': 'req-39da68a7-fd71-42c6-bc36-33b6dd49f593'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:10.992713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-39da68a7-fd71-42c6-bc36-33b6dd49f593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c47248eb06547379f35ff1ff9326f79', 'request_id': 'req-39da68a7-fd71-42c6-bc36-33b6dd49f593'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:11.004825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:11.018413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-39da68a7-fd71-42c6-bc36-33b6dd49f593 admin admin] The token cache is being invalidate because project 8c47248eb06547379f35ff1ff9326f79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:11.018884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-39da68a7-fd71-42c6-bc36-33b6dd49f593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:11.023577 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-39da68a7-fd71-42c6-bc36-33b6dd49f593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:11.025445 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 938/1885] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:10 2022] DELETE /identity/v3/projects/8c47248eb06547379f35ff1ff9326f79 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:11.030872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e55a0e-288c-49be-bbd9-a5339f27e1a0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:11.031405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e55a0e-288c-49be-bbd9-a5339f27e1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:11.031598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e55a0e-288c-49be-bbd9-a5339f27e1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:11.055993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d6e55a0e-288c-49be-bbd9-a5339f27e1a0 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:11.057259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d6e55a0e-288c-49be-bbd9-a5339f27e1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:11.131715 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 948/1886] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:11.141939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa998c-6a04-4d56-b3fa-044426d702bb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:11.142471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa998c-6a04-4d56-b3fa-044426d702bb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:11.142958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa998c-6a04-4d56-b3fa-044426d702bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:11.170727 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-65fa998c-6a04-4d56-b3fa-044426d702bb None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:11.171954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-65fa998c-6a04-4d56-b3fa-044426d702bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:11.181371 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 939/1887] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:11.185946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e55a0e-288c-49be-bbd9-a5339f27e1a0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:11.197732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e55a0e-288c-49be-bbd9-a5339f27e1a0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:11.203931 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415e41d-5d15-4ff2-ae44-d017e410e184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:11.204744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415e41d-5d15-4ff2-ae44-d017e410e184 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:11.205176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415e41d-5d15-4ff2-ae44-d017e410e184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:11.205473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415e41d-5d15-4ff2-ae44-d017e410e184 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:11.206270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415e41d-5d15-4ff2-ae44-d017e410e184 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:15:11.206585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415e41d-5d15-4ff2-ae44-d017e410e184 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:11.208366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415e41d-5d15-4ff2-ae44-d017e410e184 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:11.221649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 949/1888] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:15:11.226483 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa998c-6a04-4d56-b3fa-044426d702bb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:11.236646 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa998c-6a04-4d56-b3fa-044426d702bb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:11.242847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab296ff-dedc-40be-80b5-93f1e99e0678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:11.243472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab296ff-dedc-40be-80b5-93f1e99e0678 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:11.243749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab296ff-dedc-40be-80b5-93f1e99e0678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:11.244021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab296ff-dedc-40be-80b5-93f1e99e0678 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:11.244698 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab296ff-dedc-40be-80b5-93f1e99e0678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:11.246767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab296ff-dedc-40be-80b5-93f1e99e0678 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:11.286181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 940/1889] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:11 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:11.290085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415e41d-5d15-4ff2-ae44-d017e410e184 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:11.301288 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415e41d-5d15-4ff2-ae44-d017e410e184 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:11.365543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e49010-b0f4-40ee-84dc-76a0b70979b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:11.366082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e49010-b0f4-40ee-84dc-76a0b70979b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:11.366361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e49010-b0f4-40ee-84dc-76a0b70979b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:11.366652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e49010-b0f4-40ee-84dc-76a0b70979b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:11.367317 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e49010-b0f4-40ee-84dc-76a0b70979b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:11.369299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e49010-b0f4-40ee-84dc-76a0b70979b3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:11.395362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 950/1890] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:11 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:15:11.396637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab296ff-dedc-40be-80b5-93f1e99e0678 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:11.411473 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab296ff-dedc-40be-80b5-93f1e99e0678 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:11.417375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abaf5a2-d21e-4697-839d-bdee9b09a5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:11.417859 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaf5a2-d21e-4697-839d-bdee9b09a5bf admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:11.418131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaf5a2-d21e-4697-839d-bdee9b09a5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:11.418395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaf5a2-d21e-4697-839d-bdee9b09a5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:11.419034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abaf5a2-d21e-4697-839d-bdee9b09a5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:11.420703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abaf5a2-d21e-4697-839d-bdee9b09a5bf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:11.433243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 941/1891] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:11.437661 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e49010-b0f4-40ee-84dc-76a0b70979b3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:11.449651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e49010-b0f4-40ee-84dc-76a0b70979b3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:11.462812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db38ec-68ca-4c8a-85e9-da61af18c15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:11.463466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db38ec-68ca-4c8a-85e9-da61af18c15f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:11.463747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db38ec-68ca-4c8a-85e9-da61af18c15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:11.464023 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db38ec-68ca-4c8a-85e9-da61af18c15f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:11.465715 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db38ec-68ca-4c8a-85e9-da61af18c15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:11.466427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db38ec-68ca-4c8a-85e9-da61af18c15f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:11.472779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:11.481034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 951/1892] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:11.483999 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abaf5a2-d21e-4697-839d-bdee9b09a5bf admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:11.496072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abaf5a2-d21e-4697-839d-bdee9b09a5bf admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:11.501310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c854-cd85-4473-9168-9331cf8e14e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:11.501989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c854-cd85-4473-9168-9331cf8e14e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:11.502263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c854-cd85-4473-9168-9331cf8e14e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:11.502535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c854-cd85-4473-9168-9331cf8e14e0 admin admin] PATH_INFO: `/v3/projects/526fbf7a6c6d4d42a1b1353e80710c42/users/7b3ed1be79a84ee1bb1307d70ddf59bc/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:11.518634 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29c854-cd85-4473-9168-9331cf8e14e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=526fbf7a6c6d4d42a1b1353e80710c42, user_id=7b3ed1be79a84ee1bb1307d70ddf59bc, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:11.521870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29c854-cd85-4473-9168-9331cf8e14e0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:11.536766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 942/1893] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:11 2022] PUT /identity/v3/projects/526fbf7a6c6d4d42a1b1353e80710c42/users/7b3ed1be79a84ee1bb1307d70ddf59bc/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:11.547264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cc479-74c9-4965-8dd6-ffa29c28bc36 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:11.547551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cc479-74c9-4965-8dd6-ffa29c28bc36 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:11.547827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cc479-74c9-4965-8dd6-ffa29c28bc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:11.594730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6d4cc479-74c9-4965-8dd6-ffa29c28bc36 None None] MFA Rules not processed for user `7b3ed1be79a84ee1bb1307d70ddf59bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:11.595978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6d4cc479-74c9-4965-8dd6-ffa29c28bc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:11.668092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 952/1894] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:11.680859 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c854-cd85-4473-9168-9331cf8e14e0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:11.682589 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ba29c854-cd85-4473-9168-9331cf8e14e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:11.748127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c854-cd85-4473-9168-9331cf8e14e0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:11.751894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a36d98-b4ae-42de-b3b1-2a4ad97347cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:11.752516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a36d98-b4ae-42de-b3b1-2a4ad97347cc service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:11.752806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a36d98-b4ae-42de-b3b1-2a4ad97347cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:11.753106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a36d98-b4ae-42de-b3b1-2a4ad97347cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:11.764920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a36d98-b4ae-42de-b3b1-2a4ad97347cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:11.766560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a36d98-b4ae-42de-b3b1-2a4ad97347cc service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:11.776571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 943/1895] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:11.887014 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044fbb5-e49a-4a3b-b88f-2d956e10f0d9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:11.887415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044fbb5-e49a-4a3b-b88f-2d956e10f0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:11.887571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044fbb5-e49a-4a3b-b88f-2d956e10f0d9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:11.890068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 953/1896] 10.222.0.79 () {54 vars in 921 bytes} [Tue Sep 6 21:15:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:15:11.899913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24cd1-f752-4237-bb44-277af0fa1054 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:11.900249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24cd1-f752-4237-bb44-277af0fa1054 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:11.900590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24cd1-f752-4237-bb44-277af0fa1054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:11.923477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-12c24cd1-f752-4237-bb44-277af0fa1054 None None] MFA Rules not processed for user `12d51454d20c46d7a5e6d963a2f987cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:11.925452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-12c24cd1-f752-4237-bb44-277af0fa1054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:12.020941 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 944/1897] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Sep 6 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:12.041346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044fbb5-e49a-4a3b-b88f-2d956e10f0d9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:12.043239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a044fbb5-e49a-4a3b-b88f-2d956e10f0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:12.062797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044fbb5-e49a-4a3b-b88f-2d956e10f0d9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:12.071652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac020406-b4a2-4420-be2a-f9744bbfece6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:12.080896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac020406-b4a2-4420-be2a-f9744bbfece6 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:12.080896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac020406-b4a2-4420-be2a-f9744bbfece6 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:12.080896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac020406-b4a2-4420-be2a-f9744bbfece6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:12.088748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac020406-b4a2-4420-be2a-f9744bbfece6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:12.088748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac020406-b4a2-4420-be2a-f9744bbfece6 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:12.123246 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 954/1898] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:14.982715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c24cd1-f752-4237-bb44-277af0fa1054 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:14.988959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-12c24cd1-f752-4237-bb44-277af0fa1054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:15.024262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:15.082248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c24cd1-f752-4237-bb44-277af0fa1054 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:15.099212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:15.100287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:15.100692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:15.101084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] PATH_INFO: `/v3/users/992a7f66b2f74e71a6803bcfcce86ab5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:15.107603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=992a7f66b2f74e71a6803bcfcce86ab5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:15.110154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:15.174771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '992a7f66b2f74e71a6803bcfcce86ab5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:15.176355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '992a7f66b2f74e71a6803bcfcce86ab5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:15.194501 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '992a7f66b2f74e71a6803bcfcce86ab5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:15.211185 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '992a7f66b2f74e71a6803bcfcce86ab5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:15.211959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '992a7f66b2f74e71a6803bcfcce86ab5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:15.228894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 945/1899] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:14 2022] DELETE /identity/v3/users/992a7f66b2f74e71a6803bcfcce86ab5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:15.235933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae30f6-cccd-4bac-a0e4-aa578f055ddf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:15.236867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae30f6-cccd-4bac-a0e4-aa578f055ddf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:15.237147 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae30f6-cccd-4bac-a0e4-aa578f055ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:15.258382 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b6ae30f6-cccd-4bac-a0e4-aa578f055ddf None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:15.259619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b6ae30f6-cccd-4bac-a0e4-aa578f055ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:15.351875 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 955/1900] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:15.362208 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:15.363949 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:15.433506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054dd90-cd16-4568-9797-cd4c074199ba admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:15.439540 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7d73e-17c9-46fa-912f-7b2fa68c225a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:15.440617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d73e-17c9-46fa-912f-7b2fa68c225a service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:15.441110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d73e-17c9-46fa-912f-7b2fa68c225a service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:15.441539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d73e-17c9-46fa-912f-7b2fa68c225a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:15.453666 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7d73e-17c9-46fa-912f-7b2fa68c225a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:15.455549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7d73e-17c9-46fa-912f-7b2fa68c225a service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:15.467844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 946/1901] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:15.750464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae30f6-cccd-4bac-a0e4-aa578f055ddf None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:15.752062 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b6ae30f6-cccd-4bac-a0e4-aa578f055ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:15.765180 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae30f6-cccd-4bac-a0e4-aa578f055ddf None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:15.770357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:15.771092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:15.771290 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:15.771595 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] PATH_INFO: `/v3/projects/252880d276d2460395cc844fb42531cc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:15.773664 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=252880d276d2460395cc844fb42531cc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:15.776666 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:15.870304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '252880d276d2460395cc844fb42531cc', 'request_id': 'req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:15.881170 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '252880d276d2460395cc844fb42531cc', 'request_id': 'req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:15.890785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '252880d276d2460395cc844fb42531cc', 'request_id': 'req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:15.907490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:15.914579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] The token cache is being invalidate because project 252880d276d2460395cc844fb42531cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:15.915789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:15.916322 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:15.924624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7d73e-17c9-46fa-912f-7b2fa68c225a service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:15.929430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 956/1902] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:15 2022] DELETE /identity/v3/projects/252880d276d2460395cc844fb42531cc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:15.938746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7d73e-17c9-46fa-912f-7b2fa68c225a service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:16.028386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:16.058013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a108a01-233b-4fe3-9670-48f5c63efe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:16.064512 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108a01-233b-4fe3-9670-48f5c63efe0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:16.064959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108a01-233b-4fe3-9670-48f5c63efe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:16.065352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108a01-233b-4fe3-9670-48f5c63efe0a admin admin] PATH_INFO: `/v3/users/2914b8408d4d47d1bc5f3547ad32ed5b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:16.066897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a108a01-233b-4fe3-9670-48f5c63efe0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2914b8408d4d47d1bc5f3547ad32ed5b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:16.069196 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a108a01-233b-4fe3-9670-48f5c63efe0a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:16.124223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0a108a01-233b-4fe3-9670-48f5c63efe0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2914b8408d4d47d1bc5f3547ad32ed5b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:16.124983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0a108a01-233b-4fe3-9670-48f5c63efe0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2914b8408d4d47d1bc5f3547ad32ed5b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:16.133088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0a108a01-233b-4fe3-9670-48f5c63efe0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2914b8408d4d47d1bc5f3547ad32ed5b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:16.146543 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0a108a01-233b-4fe3-9670-48f5c63efe0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2914b8408d4d47d1bc5f3547ad32ed5b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:16.147393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0a108a01-233b-4fe3-9670-48f5c63efe0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2914b8408d4d47d1bc5f3547ad32ed5b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:16.169438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 947/1903] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:15 2022] DELETE /identity/v3/users/2914b8408d4d47d1bc5f3547ad32ed5b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:15:16.606807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:16.608721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:16.625099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8e2cc-f0fa-4322-a447-09f9f1fce0f6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:16.689854 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4176b3b-014c-432c-9153-5c390c38cc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:16.690520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4176b3b-014c-432c-9153-5c390c38cc10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:16.690793 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4176b3b-014c-432c-9153-5c390c38cc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:16.691163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4176b3b-014c-432c-9153-5c390c38cc10 admin admin] PATH_INFO: `/v3/projects/8ff70bd9b22a45dc9fe098a9f6615cfe` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:16.692330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4176b3b-014c-432c-9153-5c390c38cc10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff70bd9b22a45dc9fe098a9f6615cfe)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:16.694361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4176b3b-014c-432c-9153-5c390c38cc10 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:16.745437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4176b3b-014c-432c-9153-5c390c38cc10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff70bd9b22a45dc9fe098a9f6615cfe', 'request_id': 'req-b4176b3b-014c-432c-9153-5c390c38cc10'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:16.753939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4176b3b-014c-432c-9153-5c390c38cc10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff70bd9b22a45dc9fe098a9f6615cfe', 'request_id': 'req-b4176b3b-014c-432c-9153-5c390c38cc10'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:16.762754 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4176b3b-014c-432c-9153-5c390c38cc10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff70bd9b22a45dc9fe098a9f6615cfe', 'request_id': 'req-b4176b3b-014c-432c-9153-5c390c38cc10'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:16.777206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4176b3b-014c-432c-9153-5c390c38cc10 admin admin] The token cache is being invalidate because project 8ff70bd9b22a45dc9fe098a9f6615cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:16.777476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4176b3b-014c-432c-9153-5c390c38cc10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:16.778328 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4176b3b-014c-432c-9153-5c390c38cc10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:16.780206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 957/1904] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:16 2022] DELETE /identity/v3/projects/8ff70bd9b22a45dc9fe098a9f6615cfe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:16.785432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203840a1-e12b-4f03-a435-24394ab30d2d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:16.785721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203840a1-e12b-4f03-a435-24394ab30d2d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:16.789795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203840a1-e12b-4f03-a435-24394ab30d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:16.808117 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-203840a1-e12b-4f03-a435-24394ab30d2d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:16.809462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-203840a1-e12b-4f03-a435-24394ab30d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:16.879104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 948/1905] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:16.889164 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a474d9-aa12-4e96-ae3c-99d6c7422070 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:16.889564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a474d9-aa12-4e96-ae3c-99d6c7422070 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:16.889975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a474d9-aa12-4e96-ae3c-99d6c7422070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:16.914212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-22a474d9-aa12-4e96-ae3c-99d6c7422070 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:16.915456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-22a474d9-aa12-4e96-ae3c-99d6c7422070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:16.924916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 958/1906] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:16.929747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203840a1-e12b-4f03-a435-24394ab30d2d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:16.940222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203840a1-e12b-4f03-a435-24394ab30d2d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:16.945620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26993a42-d9f2-4c95-b01b-8d6059dc3e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:16.946396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26993a42-d9f2-4c95-b01b-8d6059dc3e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:16.946693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26993a42-d9f2-4c95-b01b-8d6059dc3e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:16.946960 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26993a42-d9f2-4c95-b01b-8d6059dc3e2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:16.947635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26993a42-d9f2-4c95-b01b-8d6059dc3e2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:15:16.947915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26993a42-d9f2-4c95-b01b-8d6059dc3e2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:16.949589 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26993a42-d9f2-4c95-b01b-8d6059dc3e2b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:16.962742 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 949/1907] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:15:16.969358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a474d9-aa12-4e96-ae3c-99d6c7422070 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:16.979278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a474d9-aa12-4e96-ae3c-99d6c7422070 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:16.985577 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33405eda-4e19-4548-ad2a-af90cbd4a72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:16.986198 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33405eda-4e19-4548-ad2a-af90cbd4a72f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:16.986478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33405eda-4e19-4548-ad2a-af90cbd4a72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:16.986742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33405eda-4e19-4548-ad2a-af90cbd4a72f admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:16.987385 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33405eda-4e19-4548-ad2a-af90cbd4a72f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:16.989481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33405eda-4e19-4548-ad2a-af90cbd4a72f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:17.019983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 959/1908] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:16 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:17.024165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26993a42-d9f2-4c95-b01b-8d6059dc3e2b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:17.034567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26993a42-d9f2-4c95-b01b-8d6059dc3e2b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:17.103991 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:17.116274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d417e45-d63d-448a-a24b-eecc6ec1deaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:17.117210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d417e45-d63d-448a-a24b-eecc6ec1deaf admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:17.117388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d417e45-d63d-448a-a24b-eecc6ec1deaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:17.117620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d417e45-d63d-448a-a24b-eecc6ec1deaf admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:17.118321 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d417e45-d63d-448a-a24b-eecc6ec1deaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:17.120947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d417e45-d63d-448a-a24b-eecc6ec1deaf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:17.147992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 950/1909] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:17 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:17.152102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33405eda-4e19-4548-ad2a-af90cbd4a72f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:17.187855 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33405eda-4e19-4548-ad2a-af90cbd4a72f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:17.199245 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec36a0-0106-4328-b154-bb0a27157092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:17.200264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec36a0-0106-4328-b154-bb0a27157092 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:17.200624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec36a0-0106-4328-b154-bb0a27157092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:17.200990 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec36a0-0106-4328-b154-bb0a27157092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:17.202533 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec36a0-0106-4328-b154-bb0a27157092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:17.204969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec36a0-0106-4328-b154-bb0a27157092 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:17.217221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 960/1910] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:17.222102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d417e45-d63d-448a-a24b-eecc6ec1deaf admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:17.236998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d417e45-d63d-448a-a24b-eecc6ec1deaf admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:17.241956 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c690d71-4479-44ab-a581-593dbbfc8b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:17.242710 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c690d71-4479-44ab-a581-593dbbfc8b82 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:17.243088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c690d71-4479-44ab-a581-593dbbfc8b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:17.243400 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c690d71-4479-44ab-a581-593dbbfc8b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:17.244314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c690d71-4479-44ab-a581-593dbbfc8b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:17.246205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c690d71-4479-44ab-a581-593dbbfc8b82 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:17.268756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec36a0-0106-4328-b154-bb0a27157092 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:17.282665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 951/1911] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:17.287091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec36a0-0106-4328-b154-bb0a27157092 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:17.298296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9690b3b-9ecd-4b6a-8f79-2947fe60632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:17.299168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690b3b-9ecd-4b6a-8f79-2947fe60632e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:17.299906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690b3b-9ecd-4b6a-8f79-2947fe60632e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:17.300307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690b3b-9ecd-4b6a-8f79-2947fe60632e admin admin] PATH_INFO: `/v3/projects/2a7edfb85235454cbd7e6e5ca6164cba/users/b50c9683c41b46f6a0c1dba39de08c08/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:17.317408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9690b3b-9ecd-4b6a-8f79-2947fe60632e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7edfb85235454cbd7e6e5ca6164cba, user_id=b50c9683c41b46f6a0c1dba39de08c08, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:17.320165 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9690b3b-9ecd-4b6a-8f79-2947fe60632e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:17.335354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 961/1912] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:17 2022] PUT /identity/v3/projects/2a7edfb85235454cbd7e6e5ca6164cba/users/b50c9683c41b46f6a0c1dba39de08c08/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:17.348244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18a70e-aa3a-45c5-ba4e-aff2627964c7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:17.348525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18a70e-aa3a-45c5-ba4e-aff2627964c7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:17.348881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18a70e-aa3a-45c5-ba4e-aff2627964c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:17.379889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cb18a70e-aa3a-45c5-ba4e-aff2627964c7 None None] MFA Rules not processed for user `b50c9683c41b46f6a0c1dba39de08c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:17.381183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cb18a70e-aa3a-45c5-ba4e-aff2627964c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:17.452838 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 952/1913] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:17.458650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9690b3b-9ecd-4b6a-8f79-2947fe60632e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:17.491880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:17.535810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9690b3b-9ecd-4b6a-8f79-2947fe60632e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:17.541917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be435b9-b666-421b-baca-3de7dd86b9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:17.542725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be435b9-b666-421b-baca-3de7dd86b9a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:17.543051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be435b9-b666-421b-baca-3de7dd86b9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:17.543408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be435b9-b666-421b-baca-3de7dd86b9a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:17.544163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be435b9-b666-421b-baca-3de7dd86b9a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:17.546138 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be435b9-b666-421b-baca-3de7dd86b9a8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:17.583574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 962/1914] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:17 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:17.588508 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18a70e-aa3a-45c5-ba4e-aff2627964c7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:17.609513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18a70e-aa3a-45c5-ba4e-aff2627964c7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:17.713934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532e89a-f47e-4a84-be0a-9e1702c62542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:17.715072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532e89a-f47e-4a84-be0a-9e1702c62542 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:17.716235 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532e89a-f47e-4a84-be0a-9e1702c62542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:17.716582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532e89a-f47e-4a84-be0a-9e1702c62542 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:17.717282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532e89a-f47e-4a84-be0a-9e1702c62542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:17.720548 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532e89a-f47e-4a84-be0a-9e1702c62542 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:17.754296 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 953/1915] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:17 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:17.758728 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be435b9-b666-421b-baca-3de7dd86b9a8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:17.773151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be435b9-b666-421b-baca-3de7dd86b9a8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:17.778772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c035a-6595-496d-9e48-09f91bef78fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:17.779470 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c035a-6595-496d-9e48-09f91bef78fa admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:17.779853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c035a-6595-496d-9e48-09f91bef78fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:17.780188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c035a-6595-496d-9e48-09f91bef78fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:17.780875 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c035a-6595-496d-9e48-09f91bef78fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:17.782510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c035a-6595-496d-9e48-09f91bef78fa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:17.794647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 963/1916] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:17.798981 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532e89a-f47e-4a84-be0a-9e1702c62542 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:17.810046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532e89a-f47e-4a84-be0a-9e1702c62542 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:17.815580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8915b-9d51-46ca-a337-66d5b7ff6b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:17.816314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8915b-9d51-46ca-a337-66d5b7ff6b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:17.816593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8915b-9d51-46ca-a337-66d5b7ff6b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:17.816898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8915b-9d51-46ca-a337-66d5b7ff6b12 admin admin] PATH_INFO: `/v3/projects/72fd77b0695e419c800b8011ad5506dd/users/c65e39718b9644a4be869375eff8d08c/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:17.832564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8915b-9d51-46ca-a337-66d5b7ff6b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72fd77b0695e419c800b8011ad5506dd, user_id=c65e39718b9644a4be869375eff8d08c, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:17.835445 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8915b-9d51-46ca-a337-66d5b7ff6b12 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:17.850032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 954/1917] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:17 2022] PUT /identity/v3/projects/72fd77b0695e419c800b8011ad5506dd/users/c65e39718b9644a4be869375eff8d08c/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:17.854169 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c035a-6595-496d-9e48-09f91bef78fa admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:17.864463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c035a-6595-496d-9e48-09f91bef78fa admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:17.929954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b8e18-892c-48ef-b866-a04858fdad68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:17.930580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b8e18-892c-48ef-b866-a04858fdad68 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:17.930857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b8e18-892c-48ef-b866-a04858fdad68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:17.931131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b8e18-892c-48ef-b866-a04858fdad68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:17.931867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b8e18-892c-48ef-b866-a04858fdad68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:17.933644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b8e18-892c-48ef-b866-a04858fdad68 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:17.948219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 964/1918] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:17.950050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8915b-9d51-46ca-a337-66d5b7ff6b12 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:17.962512 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8915b-9d51-46ca-a337-66d5b7ff6b12 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:17.967691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b3edc-e717-487b-b3a8-93f8e89f8f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:17.968411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b3edc-e717-487b-b3a8-93f8e89f8f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:17.968725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b3edc-e717-487b-b3a8-93f8e89f8f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:17.968995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b3edc-e717-487b-b3a8-93f8e89f8f9d admin admin] PATH_INFO: `/v3/projects/72fd77b0695e419c800b8011ad5506dd/users/c65e39718b9644a4be869375eff8d08c/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:17.971044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b3edc-e717-487b-b3a8-93f8e89f8f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=72fd77b0695e419c800b8011ad5506dd, user_id=c65e39718b9644a4be869375eff8d08c, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:17.974224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b3edc-e717-487b-b3a8-93f8e89f8f9d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:17.986894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 955/1919] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:17 2022] PUT /identity/v3/projects/72fd77b0695e419c800b8011ad5506dd/users/c65e39718b9644a4be869375eff8d08c/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:17.999188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f865443-03a7-4344-8102-3a670c2c0e71 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:17.999720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f865443-03a7-4344-8102-3a670c2c0e71 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:18.000626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f865443-03a7-4344-8102-3a670c2c0e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:18.041220 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7f865443-03a7-4344-8102-3a670c2c0e71 None None] MFA Rules not processed for user `c65e39718b9644a4be869375eff8d08c`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:18.042768 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7f865443-03a7-4344-8102-3a670c2c0e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:18.123989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 965/1920] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:18.138982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b3edc-e717-487b-b3a8-93f8e89f8f9d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:18.141001 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-650b3edc-e717-487b-b3a8-93f8e89f8f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:18.166314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:18.216896 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b3edc-e717-487b-b3a8-93f8e89f8f9d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:18.220635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c793d7-4a4d-447c-ab9e-37690474159b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:18.221329 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c793d7-4a4d-447c-ab9e-37690474159b service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:18.221620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c793d7-4a4d-447c-ab9e-37690474159b service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:18.221887 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c793d7-4a4d-447c-ab9e-37690474159b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:18.232355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c793d7-4a4d-447c-ab9e-37690474159b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:18.233951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c793d7-4a4d-447c-ab9e-37690474159b service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:18.244243 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 956/1921] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:18.783556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f865443-03a7-4344-8102-3a670c2c0e71 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:18.785751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7f865443-03a7-4344-8102-3a670c2c0e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:18.822457 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:18.882472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f865443-03a7-4344-8102-3a670c2c0e71 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:18.891688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385fac0-a388-4325-9c1a-425c320cc0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:18.893095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385fac0-a388-4325-9c1a-425c320cc0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:18.893544 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385fac0-a388-4325-9c1a-425c320cc0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:18.895604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385fac0-a388-4325-9c1a-425c320cc0bb admin admin] PATH_INFO: `/v3/users/5262412da2f14cd5b53bcad1b936a4bb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:18.897147 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385fac0-a388-4325-9c1a-425c320cc0bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5262412da2f14cd5b53bcad1b936a4bb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:18.901944 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385fac0-a388-4325-9c1a-425c320cc0bb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:18.919775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c793d7-4a4d-447c-ab9e-37690474159b service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:18.939133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c793d7-4a4d-447c-ab9e-37690474159b service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:18.944475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53946781-f2ed-4d90-ac38-c488fa9e2d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:18.945303 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53946781-f2ed-4d90-ac38-c488fa9e2d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:18.945580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53946781-f2ed-4d90-ac38-c488fa9e2d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:18.945942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53946781-f2ed-4d90-ac38-c488fa9e2d70 admin admin] PATH_INFO: `/v3/users/b50c9683c41b46f6a0c1dba39de08c08` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:18.947258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53946781-f2ed-4d90-ac38-c488fa9e2d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b50c9683c41b46f6a0c1dba39de08c08)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:18.949999 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53946781-f2ed-4d90-ac38-c488fa9e2d70 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:18.990789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d385fac0-a388-4325-9c1a-425c320cc0bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5262412da2f14cd5b53bcad1b936a4bb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:18.991554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d385fac0-a388-4325-9c1a-425c320cc0bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5262412da2f14cd5b53bcad1b936a4bb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:18.999820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d385fac0-a388-4325-9c1a-425c320cc0bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5262412da2f14cd5b53bcad1b936a4bb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:19.016822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d385fac0-a388-4325-9c1a-425c320cc0bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5262412da2f14cd5b53bcad1b936a4bb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:19.017833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d385fac0-a388-4325-9c1a-425c320cc0bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5262412da2f14cd5b53bcad1b936a4bb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:19.036148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 966/1922] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:18 2022] DELETE /identity/v3/users/5262412da2f14cd5b53bcad1b936a4bb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:19.038412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53946781-f2ed-4d90-ac38-c488fa9e2d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b50c9683c41b46f6a0c1dba39de08c08'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:19.040812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53946781-f2ed-4d90-ac38-c488fa9e2d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b50c9683c41b46f6a0c1dba39de08c08'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:19.043544 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168155b-7160-4f2b-a66c-996bc48af6e4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:19.044033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168155b-7160-4f2b-a66c-996bc48af6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:19.044827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168155b-7160-4f2b-a66c-996bc48af6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:19.050644 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53946781-f2ed-4d90-ac38-c488fa9e2d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b50c9683c41b46f6a0c1dba39de08c08'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:19.067308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53946781-f2ed-4d90-ac38-c488fa9e2d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b50c9683c41b46f6a0c1dba39de08c08'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:19.068155 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-53946781-f2ed-4d90-ac38-c488fa9e2d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b50c9683c41b46f6a0c1dba39de08c08'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:19.075369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2168155b-7160-4f2b-a66c-996bc48af6e4 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:19.077338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2168155b-7160-4f2b-a66c-996bc48af6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:19.090872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 957/1923] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:18 2022] DELETE /identity/v3/users/b50c9683c41b46f6a0c1dba39de08c08 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:15:19.098314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cd142-3f2c-45af-9593-05b178957e0d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:19.099379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cd142-3f2c-45af-9593-05b178957e0d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:19.099928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cd142-3f2c-45af-9593-05b178957e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:19.146699 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-389cd142-3f2c-45af-9593-05b178957e0d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:19.150105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-389cd142-3f2c-45af-9593-05b178957e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:19.187203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 967/1924] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:19.200009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168155b-7160-4f2b-a66c-996bc48af6e4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:19.202544 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2168155b-7160-4f2b-a66c-996bc48af6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:19.239765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 958/1925] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:19.254583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cd142-3f2c-45af-9593-05b178957e0d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:19.256105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-389cd142-3f2c-45af-9593-05b178957e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:19.262125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:19.281728 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168155b-7160-4f2b-a66c-996bc48af6e4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:19.310213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:19.338199 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec818fc-2753-40e5-beeb-91a9ed73c8eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:19.338819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec818fc-2753-40e5-beeb-91a9ed73c8eb service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:19.339098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec818fc-2753-40e5-beeb-91a9ed73c8eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:19.339366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec818fc-2753-40e5-beeb-91a9ed73c8eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:19.340255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cd142-3f2c-45af-9593-05b178957e0d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:19.345187 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc3d53-ac1c-4cae-bc52-134c775269a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:19.345834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc3d53-ac1c-4cae-bc52-134c775269a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:19.346112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc3d53-ac1c-4cae-bc52-134c775269a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:19.346380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc3d53-ac1c-4cae-bc52-134c775269a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:19.349402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec818fc-2753-40e5-beeb-91a9ed73c8eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:19.350954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec818fc-2753-40e5-beeb-91a9ed73c8eb service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:19.355912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc3d53-ac1c-4cae-bc52-134c775269a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:19.357390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc3d53-ac1c-4cae-bc52-134c775269a2 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:19.362104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 968/1926] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:15:19.371962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 959/1927] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:15:19.838291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec818fc-2753-40e5-beeb-91a9ed73c8eb service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:19.844695 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc3d53-ac1c-4cae-bc52-134c775269a2 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:19.846541 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1ec818fc-2753-40e5-beeb-91a9ed73c8eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:19.848906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9ffc3d53-ac1c-4cae-bc52-134c775269a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:19.905503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc3d53-ac1c-4cae-bc52-134c775269a2 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:19.909502 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec818fc-2753-40e5-beeb-91a9ed73c8eb service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:19.924614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:19.926497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:19.926784 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:19.927061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] PATH_INFO: `/v3/projects/41d2388a72d14e1fb01aa4bbffc95fa5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:19.932066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=41d2388a72d14e1fb01aa4bbffc95fa5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:19.936431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:19.938717 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:19.939216 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:19.939558 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] PATH_INFO: `/v3/projects/2a7edfb85235454cbd7e6e5ca6164cba` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:19.942648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a7edfb85235454cbd7e6e5ca6164cba)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:19.943953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:19.947349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:20.033197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41d2388a72d14e1fb01aa4bbffc95fa5', 'request_id': 'req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.050812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a7edfb85235454cbd7e6e5ca6164cba', 'request_id': 'req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.059125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a7edfb85235454cbd7e6e5ca6164cba', 'request_id': 'req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.063592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41d2388a72d14e1fb01aa4bbffc95fa5', 'request_id': 'req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.066614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a7edfb85235454cbd7e6e5ca6164cba', 'request_id': 'req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.076414 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41d2388a72d14e1fb01aa4bbffc95fa5', 'request_id': 'req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.090886 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] The token cache is being invalidate because project 2a7edfb85235454cbd7e6e5ca6164cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:20.091948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.093599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.095215 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] The token cache is being invalidate because project 41d2388a72d14e1fb01aa4bbffc95fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:20.095864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.097333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 969/1928] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:19 2022] DELETE /identity/v3/projects/2a7edfb85235454cbd7e6e5ca6164cba => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:20.101863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:20.102442 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.107599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 960/1929] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:19 2022] DELETE /identity/v3/projects/41d2388a72d14e1fb01aa4bbffc95fa5 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:20.108574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:20.109893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:20.113946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:20.227135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ff72f-dcc8-4808-a899-b6f5f43f0f86 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:20.233354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:20.234448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:20.234964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:20.235429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] PATH_INFO: `/v3/users/c65e39718b9644a4be869375eff8d08c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:20.236107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc71e9-8f7b-4f5b-a8c9-ca0bb45585dc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:20.237644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c65e39718b9644a4be869375eff8d08c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:20.240105 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:20.249262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:20.250081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:20.250361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:20.250634 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] PATH_INFO: `/v3/users/ca1a15334d7946f2b049c0cedd9f0456` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:20.255116 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca1a15334d7946f2b049c0cedd9f0456)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:20.257663 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:20.297314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:20.336016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c65e39718b9644a4be869375eff8d08c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.337707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c65e39718b9644a4be869375eff8d08c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.352813 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c65e39718b9644a4be869375eff8d08c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.377727 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca1a15334d7946f2b049c0cedd9f0456'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.381066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca1a15334d7946f2b049c0cedd9f0456'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.386163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c65e39718b9644a4be869375eff8d08c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.386163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c65e39718b9644a4be869375eff8d08c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.393010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca1a15334d7946f2b049c0cedd9f0456'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.394413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:20.403828 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 970/1930] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:20 2022] DELETE /identity/v3/users/c65e39718b9644a4be869375eff8d08c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:20.411031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca1a15334d7946f2b049c0cedd9f0456'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.411939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca1a15334d7946f2b049c0cedd9f0456'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.432268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 961/1931] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:20 2022] DELETE /identity/v3/users/ca1a15334d7946f2b049c0cedd9f0456 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:20.783258 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:20.785455 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:20.791625 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:20.793584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:20.886661 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387c523-261d-431b-8e71-0450358cadbd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:20.913060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0481a-9bbf-4e9c-a5fe-f133f29723fe admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:20.917926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:20.918583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:20.918904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:20.919182 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8 admin admin] PATH_INFO: `/v3/projects/72fd77b0695e419c800b8011ad5506dd` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:20.920308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72fd77b0695e419c800b8011ad5506dd)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:20.922289 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:20.953269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f60a7-c034-4efc-8b40-e03b0420fceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:20.954626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f60a7-c034-4efc-8b40-e03b0420fceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:20.954926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f60a7-c034-4efc-8b40-e03b0420fceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:20.955181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f60a7-c034-4efc-8b40-e03b0420fceb admin admin] PATH_INFO: `/v3/projects/c25d40777cde4bfa892b4a87cdb472f5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:20.956447 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f60a7-c034-4efc-8b40-e03b0420fceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c25d40777cde4bfa892b4a87cdb472f5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:20.958698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f60a7-c034-4efc-8b40-e03b0420fceb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:20.979660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72fd77b0695e419c800b8011ad5506dd', 'request_id': 'req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:20.991794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72fd77b0695e419c800b8011ad5506dd', 'request_id': 'req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:21.003920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72fd77b0695e419c800b8011ad5506dd', 'request_id': 'req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:21.018084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8 admin admin] The token cache is being invalidate because project 72fd77b0695e419c800b8011ad5506dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:21.018383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:21.019183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ba28efaa-f0b2-4367-9efb-67ee0fbc04d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:21.020818 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 962/1932] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:20 2022] DELETE /identity/v3/projects/72fd77b0695e419c800b8011ad5506dd => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:15:21.025614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c616e-681f-4698-a22b-da68a3103d10 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.025904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c616e-681f-4698-a22b-da68a3103d10 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.026179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c616e-681f-4698-a22b-da68a3103d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.041030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7d0f60a7-c034-4efc-8b40-e03b0420fceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c25d40777cde4bfa892b4a87cdb472f5', 'request_id': 'req-7d0f60a7-c034-4efc-8b40-e03b0420fceb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:21.044310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:21.048142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3c9c616e-681f-4698-a22b-da68a3103d10 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:21.048859 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3c9c616e-681f-4698-a22b-da68a3103d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:21.049989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7d0f60a7-c034-4efc-8b40-e03b0420fceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c25d40777cde4bfa892b4a87cdb472f5', 'request_id': 'req-7d0f60a7-c034-4efc-8b40-e03b0420fceb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:21.058841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7d0f60a7-c034-4efc-8b40-e03b0420fceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c25d40777cde4bfa892b4a87cdb472f5', 'request_id': 'req-7d0f60a7-c034-4efc-8b40-e03b0420fceb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:21.087056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7d0f60a7-c034-4efc-8b40-e03b0420fceb admin admin] The token cache is being invalidate because project c25d40777cde4bfa892b4a87cdb472f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:21.089045 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7d0f60a7-c034-4efc-8b40-e03b0420fceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:21.092589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7d0f60a7-c034-4efc-8b40-e03b0420fceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:21.096033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 971/1933] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:20 2022] DELETE /identity/v3/projects/c25d40777cde4bfa892b4a87cdb472f5 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:21.100791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f929929-fc7a-4b54-8865-cc0f19abf967 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.101075 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f929929-fc7a-4b54-8865-cc0f19abf967 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.101347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f929929-fc7a-4b54-8865-cc0f19abf967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.138202 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 963/1934] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:21.146519 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64235091-64df-4bd1-bf3c-1bb269ae0906 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.146803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64235091-64df-4bd1-bf3c-1bb269ae0906 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.147080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64235091-64df-4bd1-bf3c-1bb269ae0906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.151392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5f929929-fc7a-4b54-8865-cc0f19abf967 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:21.152726 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5f929929-fc7a-4b54-8865-cc0f19abf967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:21.157094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:21.172699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 972/1935] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:21.177677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-64235091-64df-4bd1-bf3c-1bb269ae0906 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:21.180596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-64235091-64df-4bd1-bf3c-1bb269ae0906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:21.181633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7740eb0-ba54-4337-a944-0784f905ee4b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.181918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7740eb0-ba54-4337-a944-0784f905ee4b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.185069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7740eb0-ba54-4337-a944-0784f905ee4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.193520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 964/1936] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:21.194608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64235091-64df-4bd1-bf3c-1bb269ae0906 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.205770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64235091-64df-4bd1-bf3c-1bb269ae0906 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.206792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b7740eb0-ba54-4337-a944-0784f905ee4b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:21.208013 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b7740eb0-ba54-4337-a944-0784f905ee4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:21.211508 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84581a-6811-4583-ab8e-d467cf3b96ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.212783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84581a-6811-4583-ab8e-d467cf3b96ca admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.213135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84581a-6811-4583-ab8e-d467cf3b96ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.213481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84581a-6811-4583-ab8e-d467cf3b96ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.214213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84581a-6811-4583-ab8e-d467cf3b96ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:15:21.214583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84581a-6811-4583-ab8e-d467cf3b96ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.216982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84581a-6811-4583-ab8e-d467cf3b96ca admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.220792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 973/1937] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:21.221884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7740eb0-ba54-4337-a944-0784f905ee4b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.230478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 965/1938] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:15:21.233871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7740eb0-ba54-4337-a944-0784f905ee4b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.234857 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84581a-6811-4583-ab8e-d467cf3b96ca admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.239508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9d234-7d70-4e0e-81e9-e16d405c0157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.239508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9d234-7d70-4e0e-81e9-e16d405c0157 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.240074 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9d234-7d70-4e0e-81e9-e16d405c0157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.240431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9d234-7d70-4e0e-81e9-e16d405c0157 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.242003 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9d234-7d70-4e0e-81e9-e16d405c0157 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:15:21.242721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9d234-7d70-4e0e-81e9-e16d405c0157 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.244680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9d234-7d70-4e0e-81e9-e16d405c0157 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.246018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84581a-6811-4583-ab8e-d467cf3b96ca admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.250895 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bb550-0351-416a-8c20-412ef4c65c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.251559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bb550-0351-416a-8c20-412ef4c65c8a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.251837 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bb550-0351-416a-8c20-412ef4c65c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.252106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bb550-0351-416a-8c20-412ef4c65c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.253256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bb550-0351-416a-8c20-412ef4c65c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.255086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bb550-0351-416a-8c20-412ef4c65c8a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.259445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 974/1939] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:15:21.263796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9d234-7d70-4e0e-81e9-e16d405c0157 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.276355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9d234-7d70-4e0e-81e9-e16d405c0157 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.282551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989eb40-74c8-459b-9ea1-010872f7d7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.283189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989eb40-74c8-459b-9ea1-010872f7d7bd admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.283449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989eb40-74c8-459b-9ea1-010872f7d7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.283712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989eb40-74c8-459b-9ea1-010872f7d7bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.284357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989eb40-74c8-459b-9ea1-010872f7d7bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.287192 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989eb40-74c8-459b-9ea1-010872f7d7bd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.289084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 966/1940] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:21.293695 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bb550-0351-416a-8c20-412ef4c65c8a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.323460 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 975/1941] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:15:21.327819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989eb40-74c8-459b-9ea1-010872f7d7bd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.365697 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bb550-0351-416a-8c20-412ef4c65c8a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.399503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989eb40-74c8-459b-9ea1-010872f7d7bd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.406773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c193f13-1387-4b02-a0ec-9c809e336f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.407039 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c193f13-1387-4b02-a0ec-9c809e336f03 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.407448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c193f13-1387-4b02-a0ec-9c809e336f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.407729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c193f13-1387-4b02-a0ec-9c809e336f03 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.408476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c193f13-1387-4b02-a0ec-9c809e336f03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.411128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c193f13-1387-4b02-a0ec-9c809e336f03 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.434786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b15c3c-6421-4893-a1e5-8224296df501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.435566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b15c3c-6421-4893-a1e5-8224296df501 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.435945 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b15c3c-6421-4893-a1e5-8224296df501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.436468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b15c3c-6421-4893-a1e5-8224296df501 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.437278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b15c3c-6421-4893-a1e5-8224296df501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.439497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b15c3c-6421-4893-a1e5-8224296df501 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.444626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 976/1942] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:15:21.450047 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c193f13-1387-4b02-a0ec-9c809e336f03 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.464776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c193f13-1387-4b02-a0ec-9c809e336f03 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.467439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 967/1943] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:21.470339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8d9b0-6296-4f72-a3ea-6e24664a3629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.471168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8d9b0-6296-4f72-a3ea-6e24664a3629 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.471450 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8d9b0-6296-4f72-a3ea-6e24664a3629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.471724 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8d9b0-6296-4f72-a3ea-6e24664a3629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.472542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b15c3c-6421-4893-a1e5-8224296df501 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.472681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8d9b0-6296-4f72-a3ea-6e24664a3629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.474418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8d9b0-6296-4f72-a3ea-6e24664a3629 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.485332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b15c3c-6421-4893-a1e5-8224296df501 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.486463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 977/1944] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:15:21.490615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8d9b0-6296-4f72-a3ea-6e24664a3629 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.492910 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1c69c-8dd5-473d-ad12-e95b6eaaba8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.493691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1c69c-8dd5-473d-ad12-e95b6eaaba8e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.494234 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1c69c-8dd5-473d-ad12-e95b6eaaba8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.494611 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1c69c-8dd5-473d-ad12-e95b6eaaba8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.495420 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1c69c-8dd5-473d-ad12-e95b6eaaba8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.498146 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1c69c-8dd5-473d-ad12-e95b6eaaba8e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.502319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8d9b0-6296-4f72-a3ea-6e24664a3629 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.510965 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a3542-bb31-4f6f-8040-b8a6f6eeec57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.511856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a3542-bb31-4f6f-8040-b8a6f6eeec57 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.512150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a3542-bb31-4f6f-8040-b8a6f6eeec57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.512475 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a3542-bb31-4f6f-8040-b8a6f6eeec57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.517309 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 968/1945] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:15:21.517499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a3542-bb31-4f6f-8040-b8a6f6eeec57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.517499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a3542-bb31-4f6f-8040-b8a6f6eeec57 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.518670 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1c69c-8dd5-473d-ad12-e95b6eaaba8e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.533041 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1c69c-8dd5-473d-ad12-e95b6eaaba8e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.537175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 978/1946] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:21.541362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a3542-bb31-4f6f-8040-b8a6f6eeec57 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.542365 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66954c2f-3c8a-4c31-b7ae-7ba450c5fdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.542999 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66954c2f-3c8a-4c31-b7ae-7ba450c5fdac admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.543325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66954c2f-3c8a-4c31-b7ae-7ba450c5fdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.543703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66954c2f-3c8a-4c31-b7ae-7ba450c5fdac admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.544369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66954c2f-3c8a-4c31-b7ae-7ba450c5fdac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.546067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66954c2f-3c8a-4c31-b7ae-7ba450c5fdac admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.553467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a3542-bb31-4f6f-8040-b8a6f6eeec57 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.561195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c64f66-bc33-4f16-bc9d-d1a31d84fc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.563017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c64f66-bc33-4f16-bc9d-d1a31d84fc37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.563293 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c64f66-bc33-4f16-bc9d-d1a31d84fc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.563563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c64f66-bc33-4f16-bc9d-d1a31d84fc37 admin admin] PATH_INFO: `/v3/projects/87321fcd93f84f5e9e1c97cd23118b1e/users/3f193ec725a54b08844236fd695d236a/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.568526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 969/1947] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:21.580320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66954c2f-3c8a-4c31-b7ae-7ba450c5fdac admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.601124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c64f66-bc33-4f16-bc9d-d1a31d84fc37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87321fcd93f84f5e9e1c97cd23118b1e, user_id=3f193ec725a54b08844236fd695d236a, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.603829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c64f66-bc33-4f16-bc9d-d1a31d84fc37 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.610008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66954c2f-3c8a-4c31-b7ae-7ba450c5fdac admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.621340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12722870-cf74-459e-ba79-fdec3786f860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.622130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12722870-cf74-459e-ba79-fdec3786f860 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.622407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12722870-cf74-459e-ba79-fdec3786f860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.622679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12722870-cf74-459e-ba79-fdec3786f860 admin admin] PATH_INFO: `/v3/projects/aa4a7a104c2640d4911fb54c69fbef0b/users/3944ef4b57ea438490bbf5ea29fa15e6/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.624131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 979/1948] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:21 2022] PUT /identity/v3/projects/87321fcd93f84f5e9e1c97cd23118b1e/users/3f193ec725a54b08844236fd695d236a/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:21.635212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbf9be-0918-46a8-881f-5bcf98092cc3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.635839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbf9be-0918-46a8-881f-5bcf98092cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.636188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbf9be-0918-46a8-881f-5bcf98092cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.639679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12722870-cf74-459e-ba79-fdec3786f860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4a7a104c2640d4911fb54c69fbef0b, user_id=3944ef4b57ea438490bbf5ea29fa15e6, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.642631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12722870-cf74-459e-ba79-fdec3786f860 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.657400 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 970/1949] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:21 2022] PUT /identity/v3/projects/aa4a7a104c2640d4911fb54c69fbef0b/users/3944ef4b57ea438490bbf5ea29fa15e6/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:21.669149 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa1c91-dbb4-4037-bac6-29f2616efae2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.669449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa1c91-dbb4-4037-bac6-29f2616efae2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.669677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa1c91-dbb4-4037-bac6-29f2616efae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.669820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-96dbf9be-0918-46a8-881f-5bcf98092cc3 None None] MFA Rules not processed for user `3f193ec725a54b08844236fd695d236a`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:21.671086 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-96dbf9be-0918-46a8-881f-5bcf98092cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:21.696303 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:21.696861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:21.699391 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b3aa1c91-dbb4-4037-bac6-29f2616efae2 None None] MFA Rules not processed for user `3944ef4b57ea438490bbf5ea29fa15e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:21.701537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b3aa1c91-dbb4-4037-bac6-29f2616efae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:21.744208 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 980/1950] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:21.748980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbf9be-0918-46a8-881f-5bcf98092cc3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.773533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 971/1951] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:21.778366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa1c91-dbb4-4037-bac6-29f2616efae2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.814604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbf9be-0918-46a8-881f-5bcf98092cc3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.819350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051fd7b-3e3b-44c9-b4bc-70464708c65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.819971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051fd7b-3e3b-44c9-b4bc-70464708c65c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.820248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051fd7b-3e3b-44c9-b4bc-70464708c65c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.820518 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051fd7b-3e3b-44c9-b4bc-70464708c65c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.821206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051fd7b-3e3b-44c9-b4bc-70464708c65c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.823143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051fd7b-3e3b-44c9-b4bc-70464708c65c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.848896 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa1c91-dbb4-4037-bac6-29f2616efae2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.855614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 981/1952] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:15:21.855823 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a619fd-bc07-4f3e-8001-fe69ea205d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:21.856915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a619fd-bc07-4f3e-8001-fe69ea205d3d admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:21.857070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051fd7b-3e3b-44c9-b4bc-70464708c65c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.857565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a619fd-bc07-4f3e-8001-fe69ea205d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:21.858181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a619fd-bc07-4f3e-8001-fe69ea205d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:21.859140 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a619fd-bc07-4f3e-8001-fe69ea205d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:21.862269 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a619fd-bc07-4f3e-8001-fe69ea205d3d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:21.901921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 972/1953] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:15:21.905799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a619fd-bc07-4f3e-8001-fe69ea205d3d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:21.928783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051fd7b-3e3b-44c9-b4bc-70464708c65c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.993883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a619fd-bc07-4f3e-8001-fe69ea205d3d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:21.996639 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:22.010278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc03cc7-fab2-4e2a-8966-fa7409aba780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.011050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc03cc7-fab2-4e2a-8966-fa7409aba780 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.011189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4b144-cc5e-4989-9766-e113427bcb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.011480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc03cc7-fab2-4e2a-8966-fa7409aba780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.011820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc03cc7-fab2-4e2a-8966-fa7409aba780 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.011960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4b144-cc5e-4989-9766-e113427bcb7f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.012302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4b144-cc5e-4989-9766-e113427bcb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.012631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4b144-cc5e-4989-9766-e113427bcb7f admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.012827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc03cc7-fab2-4e2a-8966-fa7409aba780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:22.013409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a4b144-cc5e-4989-9766-e113427bcb7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:22.015425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc03cc7-fab2-4e2a-8966-fa7409aba780 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:22.015541 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a4b144-cc5e-4989-9766-e113427bcb7f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:22.041549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 973/1954] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:15:22.042263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 982/1955] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:21 2022] POST /identity/v3/users => generated 294 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:22.046535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc03cc7-fab2-4e2a-8966-fa7409aba780 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:22.046702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4b144-cc5e-4989-9766-e113427bcb7f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:22.057882 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc03cc7-fab2-4e2a-8966-fa7409aba780 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:22.058008 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4b144-cc5e-4989-9766-e113427bcb7f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:22.062769 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986bdae-b689-4be8-9a11-53022f73fa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.063520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986bdae-b689-4be8-9a11-53022f73fa45 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.063819 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d42497-675c-4e66-ade2-1a12def32fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.063935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986bdae-b689-4be8-9a11-53022f73fa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.064191 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986bdae-b689-4be8-9a11-53022f73fa45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.064438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d42497-675c-4e66-ade2-1a12def32fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.064750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d42497-675c-4e66-ade2-1a12def32fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.065023 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d42497-675c-4e66-ade2-1a12def32fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.065113 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986bdae-b689-4be8-9a11-53022f73fa45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:22.065676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d42497-675c-4e66-ade2-1a12def32fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:22.066843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986bdae-b689-4be8-9a11-53022f73fa45 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:22.067349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d42497-675c-4e66-ade2-1a12def32fb9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:22.078764 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 974/1956] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:22.083228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d42497-675c-4e66-ade2-1a12def32fb9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:22.087102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 983/1957] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:22.091284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986bdae-b689-4be8-9a11-53022f73fa45 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:22.094200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d42497-675c-4e66-ade2-1a12def32fb9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:22.101297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51a0c2-26d0-4ff5-8f84-58fc2554930a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.101762 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986bdae-b689-4be8-9a11-53022f73fa45 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:22.101912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51a0c2-26d0-4ff5-8f84-58fc2554930a admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.102188 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51a0c2-26d0-4ff5-8f84-58fc2554930a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.102456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51a0c2-26d0-4ff5-8f84-58fc2554930a admin admin] PATH_INFO: `/v3/projects/2906957ce5f543488c83e45e6f5f2a0b/users/81d4090fdcc64ac38813913d2d52c6e8/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.104940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:22.108440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c8c76-8d19-4a44-92fe-9b7b6a99842c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.109167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c8c76-8d19-4a44-92fe-9b7b6a99842c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.109448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c8c76-8d19-4a44-92fe-9b7b6a99842c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.109723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c8c76-8d19-4a44-92fe-9b7b6a99842c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.110376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c8c76-8d19-4a44-92fe-9b7b6a99842c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:22.112253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c8c76-8d19-4a44-92fe-9b7b6a99842c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:22.120875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51a0c2-26d0-4ff5-8f84-58fc2554930a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2906957ce5f543488c83e45e6f5f2a0b, user_id=81d4090fdcc64ac38813913d2d52c6e8, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:22.123560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51a0c2-26d0-4ff5-8f84-58fc2554930a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:22.124222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 984/1958] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:15:22.129005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c8c76-8d19-4a44-92fe-9b7b6a99842c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:22.141951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 975/1959] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:22 2022] PUT /identity/v3/projects/2906957ce5f543488c83e45e6f5f2a0b/users/81d4090fdcc64ac38813913d2d52c6e8/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:22.142602 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c8c76-8d19-4a44-92fe-9b7b6a99842c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:22.143112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51a0c2-26d0-4ff5-8f84-58fc2554930a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:22.213615 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51a0c2-26d0-4ff5-8f84-58fc2554930a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:22.222462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869bded-e5cc-4546-83d1-09138737c4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.223302 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869bded-e5cc-4546-83d1-09138737c4ff admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.223771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869bded-e5cc-4546-83d1-09138737c4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.224625 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869bded-e5cc-4546-83d1-09138737c4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.226075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869bded-e5cc-4546-83d1-09138737c4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:22.228495 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869bded-e5cc-4546-83d1-09138737c4ff admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:22.239013 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70085ead-8e8e-4ddc-8354-316bc23adc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.240095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70085ead-8e8e-4ddc-8354-316bc23adc2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.240463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70085ead-8e8e-4ddc-8354-316bc23adc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.240838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70085ead-8e8e-4ddc-8354-316bc23adc2f admin admin] PATH_INFO: `/v3/projects/f14ae0e55da249dba89961f0339fd655/users/e3e3f325e24a475ca817c737a65ea553/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.247302 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 976/1960] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:22.248819 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869bded-e5cc-4546-83d1-09138737c4ff admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:22.260500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70085ead-8e8e-4ddc-8354-316bc23adc2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14ae0e55da249dba89961f0339fd655, user_id=e3e3f325e24a475ca817c737a65ea553, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:22.261184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869bded-e5cc-4546-83d1-09138737c4ff admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:22.263812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70085ead-8e8e-4ddc-8354-316bc23adc2f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:22.266306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf245c-0510-422a-b4a7-49bf4bcab77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.266994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf245c-0510-422a-b4a7-49bf4bcab77c admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.267273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf245c-0510-422a-b4a7-49bf4bcab77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.267550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf245c-0510-422a-b4a7-49bf4bcab77c admin admin] PATH_INFO: `/v3/projects/2906957ce5f543488c83e45e6f5f2a0b/users/81d4090fdcc64ac38813913d2d52c6e8/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.272254 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf245c-0510-422a-b4a7-49bf4bcab77c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2906957ce5f543488c83e45e6f5f2a0b, user_id=81d4090fdcc64ac38813913d2d52c6e8, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:22.275523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf245c-0510-422a-b4a7-49bf4bcab77c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:22.282527 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 985/1961] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:22 2022] PUT /identity/v3/projects/f14ae0e55da249dba89961f0339fd655/users/e3e3f325e24a475ca817c737a65ea553/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:22.295163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e028d-222f-4848-af0d-078dd16aad85 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.295446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e028d-222f-4848-af0d-078dd16aad85 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.295756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e028d-222f-4848-af0d-078dd16aad85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.298490 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 977/1962] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:22 2022] PUT /identity/v3/projects/2906957ce5f543488c83e45e6f5f2a0b/users/81d4090fdcc64ac38813913d2d52c6e8/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:22.311003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79822ba1-6bfc-441c-9340-e15efa08daf8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.311003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79822ba1-6bfc-441c-9340-e15efa08daf8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.311324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79822ba1-6bfc-441c-9340-e15efa08daf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.342465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f45e028d-222f-4848-af0d-078dd16aad85 None None] MFA Rules not processed for user `e3e3f325e24a475ca817c737a65ea553`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:22.343891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f45e028d-222f-4848-af0d-078dd16aad85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:22.358612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-79822ba1-6bfc-441c-9340-e15efa08daf8 None None] MFA Rules not processed for user `81d4090fdcc64ac38813913d2d52c6e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:22.360828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-79822ba1-6bfc-441c-9340-e15efa08daf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:22.464851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 978/1963] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:22.469822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 986/1964] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:22.473339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79822ba1-6bfc-441c-9340-e15efa08daf8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:22.474868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-79822ba1-6bfc-441c-9340-e15efa08daf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:22.476445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e028d-222f-4848-af0d-078dd16aad85 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:22.540778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:22.550641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79822ba1-6bfc-441c-9340-e15efa08daf8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:22.555860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5387d3-34af-46a2-b4c8-66708a7d7024 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.557263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5387d3-34af-46a2-b4c8-66708a7d7024 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.557675 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5387d3-34af-46a2-b4c8-66708a7d7024 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.558117 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5387d3-34af-46a2-b4c8-66708a7d7024 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.572789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:22.578073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5387d3-34af-46a2-b4c8-66708a7d7024 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:22.580386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5387d3-34af-46a2-b4c8-66708a7d7024 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:22.581144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e028d-222f-4848-af0d-078dd16aad85 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:22.596105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 979/1965] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:22.601732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052071df-72a6-444d-9567-f817bdd5749b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.603750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052071df-72a6-444d-9567-f817bdd5749b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.604131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052071df-72a6-444d-9567-f817bdd5749b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.604521 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052071df-72a6-444d-9567-f817bdd5749b admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.605463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052071df-72a6-444d-9567-f817bdd5749b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:22.607683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052071df-72a6-444d-9567-f817bdd5749b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:22.657993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 987/1966] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:22 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:22.662726 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5387d3-34af-46a2-b4c8-66708a7d7024 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:22.679715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5387d3-34af-46a2-b4c8-66708a7d7024 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:22.790524 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052071df-72a6-444d-9567-f817bdd5749b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:22.793268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-052071df-72a6-444d-9567-f817bdd5749b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:22.878024 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6380b-8860-476a-99bf-3a878274e2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.879354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6380b-8860-476a-99bf-3a878274e2f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.879780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6380b-8860-476a-99bf-3a878274e2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.880222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6380b-8860-476a-99bf-3a878274e2f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.881211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6380b-8860-476a-99bf-3a878274e2f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:22.883820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6380b-8860-476a-99bf-3a878274e2f4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:22.923065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 980/1967] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:22 2022] POST /identity/v3/users => generated 300 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:22.923942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6380b-8860-476a-99bf-3a878274e2f4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:22.940111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052071df-72a6-444d-9567-f817bdd5749b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:22.945723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c2017-d8d5-4b4f-b73e-b40be2e81988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.946392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2017-d8d5-4b4f-b73e-b40be2e81988 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.947224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2017-d8d5-4b4f-b73e-b40be2e81988 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.947224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2017-d8d5-4b4f-b73e-b40be2e81988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.951450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6380b-8860-476a-99bf-3a878274e2f4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:22.959257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114da82-16fe-4eb3-9d67-2c42f156ca2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:22.959897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114da82-16fe-4eb3-9d67-2c42f156ca2f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:22.960548 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114da82-16fe-4eb3-9d67-2c42f156ca2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:22.960869 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114da82-16fe-4eb3-9d67-2c42f156ca2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:22.961638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114da82-16fe-4eb3-9d67-2c42f156ca2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:22.963314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114da82-16fe-4eb3-9d67-2c42f156ca2f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:22.976121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 981/1968] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:22.981434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114da82-16fe-4eb3-9d67-2c42f156ca2f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:22.994425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114da82-16fe-4eb3-9d67-2c42f156ca2f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:23.002506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c581b4-6552-4b8a-891a-7f1aa466aab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:23.003492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c581b4-6552-4b8a-891a-7f1aa466aab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:23.003844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c581b4-6552-4b8a-891a-7f1aa466aab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:23.004180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c581b4-6552-4b8a-891a-7f1aa466aab0 admin admin] PATH_INFO: `/v3/projects/9ec4174aacf64c548e53ccde9d24cd06/users/ddba77508406430f920ab8e54429b6a2/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:23.021357 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c581b4-6552-4b8a-891a-7f1aa466aab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec4174aacf64c548e53ccde9d24cd06, user_id=ddba77508406430f920ab8e54429b6a2, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:23.026081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c581b4-6552-4b8a-891a-7f1aa466aab0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:23.027812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c2017-d8d5-4b4f-b73e-b40be2e81988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:23.029893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c2017-d8d5-4b4f-b73e-b40be2e81988 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:23.046111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 988/1970] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:23.052631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 982/1970] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:22 2022] PUT /identity/v3/projects/9ec4174aacf64c548e53ccde9d24cd06/users/ddba77508406430f920ab8e54429b6a2/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:15:23.053211 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c2017-d8d5-4b4f-b73e-b40be2e81988 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:23.065301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c2017-d8d5-4b4f-b73e-b40be2e81988 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:23.169199 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e68d98-7df8-4b92-b954-fcbbd6f4c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:23.170433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e68d98-7df8-4b92-b954-fcbbd6f4c442 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:23.170766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e68d98-7df8-4b92-b954-fcbbd6f4c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:23.171191 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e68d98-7df8-4b92-b954-fcbbd6f4c442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:23.172716 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e68d98-7df8-4b92-b954-fcbbd6f4c442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:23.174098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e68d98-7df8-4b92-b954-fcbbd6f4c442 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:23.188559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 989/1971] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:23.193049 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c581b4-6552-4b8a-891a-7f1aa466aab0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:23.207409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c581b4-6552-4b8a-891a-7f1aa466aab0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:23.212622 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32663acf-9a16-4c42-8a97-3edd6f9bb3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:23.213971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663acf-9a16-4c42-8a97-3edd6f9bb3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:23.214255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663acf-9a16-4c42-8a97-3edd6f9bb3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:23.214536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663acf-9a16-4c42-8a97-3edd6f9bb3ea admin admin] PATH_INFO: `/v3/projects/9ec4174aacf64c548e53ccde9d24cd06/users/ddba77508406430f920ab8e54429b6a2/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:23.216692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32663acf-9a16-4c42-8a97-3edd6f9bb3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec4174aacf64c548e53ccde9d24cd06, user_id=ddba77508406430f920ab8e54429b6a2, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:23.219552 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32663acf-9a16-4c42-8a97-3edd6f9bb3ea admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:23.232015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 983/1972] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:23 2022] PUT /identity/v3/projects/9ec4174aacf64c548e53ccde9d24cd06/users/ddba77508406430f920ab8e54429b6a2/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:15:23.243673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488fa67-3c8c-4e98-9e77-38ef778324bd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:23.243961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488fa67-3c8c-4e98-9e77-38ef778324bd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:23.244233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488fa67-3c8c-4e98-9e77-38ef778324bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:23.279742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:23.284198 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b488fa67-3c8c-4e98-9e77-38ef778324bd None None] MFA Rules not processed for user `ddba77508406430f920ab8e54429b6a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:23.287058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b488fa67-3c8c-4e98-9e77-38ef778324bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:23.368746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 990/1973] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:23.379004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32663acf-9a16-4c42-8a97-3edd6f9bb3ea admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:23.453810 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32663acf-9a16-4c42-8a97-3edd6f9bb3ea admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:23.460390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da6847-b9b2-4f42-b3b2-df834bceabd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:23.461129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da6847-b9b2-4f42-b3b2-df834bceabd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:23.461499 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da6847-b9b2-4f42-b3b2-df834bceabd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:23.462213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da6847-b9b2-4f42-b3b2-df834bceabd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:23.474066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da6847-b9b2-4f42-b3b2-df834bceabd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:23.475942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da6847-b9b2-4f42-b3b2-df834bceabd7 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:23.493954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 984/1974] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:24.152466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488fa67-3c8c-4e98-9e77-38ef778324bd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:24.162682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488fa67-3c8c-4e98-9e77-38ef778324bd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:24.166463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26616661-fe33-42a1-ba6f-b852c2a7d524 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:24.167051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26616661-fe33-42a1-ba6f-b852c2a7d524 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:24.167324 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26616661-fe33-42a1-ba6f-b852c2a7d524 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:24.167599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26616661-fe33-42a1-ba6f-b852c2a7d524 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:24.230910 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26616661-fe33-42a1-ba6f-b852c2a7d524 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:24.232390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26616661-fe33-42a1-ba6f-b852c2a7d524 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:24.241500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 991/1975] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:33.484758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:33.657749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da6847-b9b2-4f42-b3b2-df834bceabd7 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:33.783121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da6847-b9b2-4f42-b3b2-df834bceabd7 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:33.787951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6a1a8-fcd8-4fef-8e1f-90c9b138dc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:33.788500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6a1a8-fcd8-4fef-8e1f-90c9b138dc57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:33.788835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6a1a8-fcd8-4fef-8e1f-90c9b138dc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:33.789143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6a1a8-fcd8-4fef-8e1f-90c9b138dc57 admin admin] PATH_INFO: `/v3/users/3944ef4b57ea438490bbf5ea29fa15e6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:33.790286 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6a1a8-fcd8-4fef-8e1f-90c9b138dc57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3944ef4b57ea438490bbf5ea29fa15e6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:33.792222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6a1a8-fcd8-4fef-8e1f-90c9b138dc57 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:33.862871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c4d6a1a8-fcd8-4fef-8e1f-90c9b138dc57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3944ef4b57ea438490bbf5ea29fa15e6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:33.863940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c4d6a1a8-fcd8-4fef-8e1f-90c9b138dc57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3944ef4b57ea438490bbf5ea29fa15e6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:33.873760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c4d6a1a8-fcd8-4fef-8e1f-90c9b138dc57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3944ef4b57ea438490bbf5ea29fa15e6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:33.891766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c4d6a1a8-fcd8-4fef-8e1f-90c9b138dc57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3944ef4b57ea438490bbf5ea29fa15e6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:33.892287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c4d6a1a8-fcd8-4fef-8e1f-90c9b138dc57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3944ef4b57ea438490bbf5ea29fa15e6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:33.906907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26616661-fe33-42a1-ba6f-b852c2a7d524 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:33.910581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 985/1976] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:33 2022] DELETE /identity/v3/users/3944ef4b57ea438490bbf5ea29fa15e6 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:33.915118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611961d7-6042-48f9-bd4e-423faaf06c32 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:33.915403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611961d7-6042-48f9-bd4e-423faaf06c32 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:33.915758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611961d7-6042-48f9-bd4e-423faaf06c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:33.917251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26616661-fe33-42a1-ba6f-b852c2a7d524 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:33.938386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-611961d7-6042-48f9-bd4e-423faaf06c32 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:33.939957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-611961d7-6042-48f9-bd4e-423faaf06c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:33.981623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d729fc-743d-41bf-bcea-7b125c1a59b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:33.982415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d729fc-743d-41bf-bcea-7b125c1a59b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:33.982832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d729fc-743d-41bf-bcea-7b125c1a59b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:33.983220 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d729fc-743d-41bf-bcea-7b125c1a59b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:33.985662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-98d729fc-743d-41bf-bcea-7b125c1a59b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:34.036483 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 986/1977] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:34.044176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611961d7-6042-48f9-bd4e-423faaf06c32 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:34.045966 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-611961d7-6042-48f9-bd4e-423faaf06c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:34.076217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:34.084068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:34.095872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d729fc-743d-41bf-bcea-7b125c1a59b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:34.097408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d729fc-743d-41bf-bcea-7b125c1a59b0 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:34.107975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 992/1978] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:15:34.127575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611961d7-6042-48f9-bd4e-423faaf06c32 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:34.132718 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b01757-5e06-4141-8b9a-f2f4f070d94f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:34.133628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b01757-5e06-4141-8b9a-f2f4f070d94f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:34.134063 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b01757-5e06-4141-8b9a-f2f4f070d94f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:34.134546 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b01757-5e06-4141-8b9a-f2f4f070d94f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:34.146429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b01757-5e06-4141-8b9a-f2f4f070d94f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:34.148143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b01757-5e06-4141-8b9a-f2f4f070d94f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:34.160361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 987/1979] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:15:34.471927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d729fc-743d-41bf-bcea-7b125c1a59b0 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:34.474146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-98d729fc-743d-41bf-bcea-7b125c1a59b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:34.493482 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d729fc-743d-41bf-bcea-7b125c1a59b0 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:34.499559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:34.500301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:34.505317 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:34.505610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] PATH_INFO: `/v3/projects/aa4a7a104c2640d4911fb54c69fbef0b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:34.507963 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4a7a104c2640d4911fb54c69fbef0b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:34.515274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:34.584314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4a7a104c2640d4911fb54c69fbef0b', 'request_id': 'req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:34.597311 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4a7a104c2640d4911fb54c69fbef0b', 'request_id': 'req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:34.609354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4a7a104c2640d4911fb54c69fbef0b', 'request_id': 'req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:34.631110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] The token cache is being invalidate because project aa4a7a104c2640d4911fb54c69fbef0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:34.631554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:34.632545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:34.635634 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 993/1980] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:34 2022] DELETE /identity/v3/projects/aa4a7a104c2640d4911fb54c69fbef0b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:34.640962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b01757-5e06-4141-8b9a-f2f4f070d94f service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:34.653060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b01757-5e06-4141-8b9a-f2f4f070d94f service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:34.744216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:34.745104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:34.747079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:34.747539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] PATH_INFO: `/v3/users/81d4090fdcc64ac38813913d2d52c6e8` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:34.749779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d4090fdcc64ac38813913d2d52c6e8)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:34.753413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:34.829796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d4090fdcc64ac38813913d2d52c6e8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:34.830776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d4090fdcc64ac38813913d2d52c6e8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:34.842164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d4090fdcc64ac38813913d2d52c6e8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:34.857362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d4090fdcc64ac38813913d2d52c6e8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:34.858131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d4090fdcc64ac38813913d2d52c6e8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:34.873560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 988/1981] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:34 2022] DELETE /identity/v3/users/81d4090fdcc64ac38813913d2d52c6e8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:35.072550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:35.074824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:35.091838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b8d5f-bc6b-4b35-ad74-aba39a3d21df admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:35.175786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b2f26-d766-4b0b-a3ff-13756ddd6a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:35.176539 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b2f26-d766-4b0b-a3ff-13756ddd6a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:35.176915 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b2f26-d766-4b0b-a3ff-13756ddd6a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:35.177246 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b2f26-d766-4b0b-a3ff-13756ddd6a38 admin admin] PATH_INFO: `/v3/users/4997f3e09ed14a5f89c5815bd8ee7255` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:35.178867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b2f26-d766-4b0b-a3ff-13756ddd6a38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4997f3e09ed14a5f89c5815bd8ee7255)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:35.181364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b2f26-d766-4b0b-a3ff-13756ddd6a38 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:35.270282 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae7b2f26-d766-4b0b-a3ff-13756ddd6a38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4997f3e09ed14a5f89c5815bd8ee7255'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:35.271020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae7b2f26-d766-4b0b-a3ff-13756ddd6a38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4997f3e09ed14a5f89c5815bd8ee7255'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:35.281171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae7b2f26-d766-4b0b-a3ff-13756ddd6a38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4997f3e09ed14a5f89c5815bd8ee7255'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:35.283688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:35.286037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:35.302646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae7b2f26-d766-4b0b-a3ff-13756ddd6a38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4997f3e09ed14a5f89c5815bd8ee7255'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:35.303521 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae7b2f26-d766-4b0b-a3ff-13756ddd6a38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4997f3e09ed14a5f89c5815bd8ee7255'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:35.310897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05b6b9-1e5d-442d-baa3-cb641e3c9298 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:35.316603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:35.317384 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:35.317612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:35.317884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef admin admin] PATH_INFO: `/v3/projects/2906957ce5f543488c83e45e6f5f2a0b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:35.319259 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2906957ce5f543488c83e45e6f5f2a0b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:35.321627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:35.324827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:35.344092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 994/1982] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:35 2022] DELETE /identity/v3/users/4997f3e09ed14a5f89c5815bd8ee7255 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:35.349051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddba1d-6661-4f87-b0dd-8f9bc6442d8d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:35.349774 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddba1d-6661-4f87-b0dd-8f9bc6442d8d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:35.349774 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddba1d-6661-4f87-b0dd-8f9bc6442d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:35.379438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:35.380489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d9ddba1d-6661-4f87-b0dd-8f9bc6442d8d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:35.381762 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d9ddba1d-6661-4f87-b0dd-8f9bc6442d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:35.391053 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2906957ce5f543488c83e45e6f5f2a0b', 'request_id': 'req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:35.404690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2906957ce5f543488c83e45e6f5f2a0b', 'request_id': 'req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:35.416384 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2906957ce5f543488c83e45e6f5f2a0b', 'request_id': 'req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:35.431687 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef admin admin] The token cache is being invalidate because project 2906957ce5f543488c83e45e6f5f2a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:35.431977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:35.432641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-26ed1002-a441-4fc3-a0f6-a57f235fc6ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:35.434478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 989/1983] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:35 2022] DELETE /identity/v3/projects/2906957ce5f543488c83e45e6f5f2a0b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:15:35.439272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af37d38-1797-4459-bccc-0679d1f4326a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:35.439554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af37d38-1797-4459-bccc-0679d1f4326a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:35.439835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af37d38-1797-4459-bccc-0679d1f4326a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:35.455097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 995/1984] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:35.464639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7af37d38-1797-4459-bccc-0679d1f4326a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:35.465738 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7af37d38-1797-4459-bccc-0679d1f4326a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:35.465903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddba1d-6661-4f87-b0dd-8f9bc6442d8d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:35.467139 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d9ddba1d-6661-4f87-b0dd-8f9bc6442d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:35.475124 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 990/1985] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:35.485152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a1d95-bfc8-41af-82c6-cece4fe47242 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:35.485438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a1d95-bfc8-41af-82c6-cece4fe47242 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:35.485796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a1d95-bfc8-41af-82c6-cece4fe47242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:35.508479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-639a1d95-bfc8-41af-82c6-cece4fe47242 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:35.509647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-639a1d95-bfc8-41af-82c6-cece4fe47242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:35.517926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 991/1986] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:35.523203 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a1d95-bfc8-41af-82c6-cece4fe47242 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:35.532432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a1d95-bfc8-41af-82c6-cece4fe47242 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:35.534418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddba1d-6661-4f87-b0dd-8f9bc6442d8d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:35.537376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25e68c-ae7f-4347-9434-baa974f9bff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:35.537994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25e68c-ae7f-4347-9434-baa974f9bff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:35.538272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25e68c-ae7f-4347-9434-baa974f9bff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:35.538540 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25e68c-ae7f-4347-9434-baa974f9bff9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:35.538642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4ac2c-8e22-4f24-a51e-2fd3c8a32902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:35.539430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25e68c-ae7f-4347-9434-baa974f9bff9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:15:35.539602 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4ac2c-8e22-4f24-a51e-2fd3c8a32902 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:35.539815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25e68c-ae7f-4347-9434-baa974f9bff9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:35.539891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4ac2c-8e22-4f24-a51e-2fd3c8a32902 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:35.540240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4ac2c-8e22-4f24-a51e-2fd3c8a32902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:35.541484 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25e68c-ae7f-4347-9434-baa974f9bff9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:35.552496 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4ac2c-8e22-4f24-a51e-2fd3c8a32902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:35.554115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4ac2c-8e22-4f24-a51e-2fd3c8a32902 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:35.558082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 992/1987] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:15:35.559158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25e68c-ae7f-4347-9434-baa974f9bff9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:35.565093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 996/1988] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:15:35.570411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25e68c-ae7f-4347-9434-baa974f9bff9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:35.576343 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581d42f-8058-4e0f-a31b-cee7b690386a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:35.576994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581d42f-8058-4e0f-a31b-cee7b690386a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:35.577274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581d42f-8058-4e0f-a31b-cee7b690386a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:35.578103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581d42f-8058-4e0f-a31b-cee7b690386a admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:35.578778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581d42f-8058-4e0f-a31b-cee7b690386a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:35.580357 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581d42f-8058-4e0f-a31b-cee7b690386a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:35.619785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 993/1989] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:35 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:35.624035 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4ac2c-8e22-4f24-a51e-2fd3c8a32902 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:35.639699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4ac2c-8e22-4f24-a51e-2fd3c8a32902 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:35.700606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d066bc-1348-4754-8110-2e7c47150a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:35.701425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d066bc-1348-4754-8110-2e7c47150a21 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:35.701715 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d066bc-1348-4754-8110-2e7c47150a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:35.701997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d066bc-1348-4754-8110-2e7c47150a21 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:35.702632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d066bc-1348-4754-8110-2e7c47150a21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:35.704897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d066bc-1348-4754-8110-2e7c47150a21 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:35.732428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 997/1990] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:35 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:35.733790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581d42f-8058-4e0f-a31b-cee7b690386a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:35.745052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581d42f-8058-4e0f-a31b-cee7b690386a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:35.750797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7661b93-81f6-4920-99b5-9491c7fdbfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:35.751447 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7661b93-81f6-4920-99b5-9491c7fdbfd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:35.751756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7661b93-81f6-4920-99b5-9491c7fdbfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:35.752163 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7661b93-81f6-4920-99b5-9491c7fdbfd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:35.752878 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7661b93-81f6-4920-99b5-9491c7fdbfd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:35.754541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7661b93-81f6-4920-99b5-9491c7fdbfd7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:35.766734 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 994/1991] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:35.771971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d066bc-1348-4754-8110-2e7c47150a21 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:35.782470 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d066bc-1348-4754-8110-2e7c47150a21 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:35.787541 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7203db-cc84-45cc-982b-3172f54d98d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:35.788349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7203db-cc84-45cc-982b-3172f54d98d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:35.788759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7203db-cc84-45cc-982b-3172f54d98d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:35.789106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7203db-cc84-45cc-982b-3172f54d98d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:35.789827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7203db-cc84-45cc-982b-3172f54d98d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:35.791490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7203db-cc84-45cc-982b-3172f54d98d3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:35.803157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 998/1992] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:35.808320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7661b93-81f6-4920-99b5-9491c7fdbfd7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:35.818993 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7661b93-81f6-4920-99b5-9491c7fdbfd7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:35.823955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98257efd-bc57-458e-91c4-21be930f227a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:35.824778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98257efd-bc57-458e-91c4-21be930f227a admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:35.825190 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98257efd-bc57-458e-91c4-21be930f227a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:35.825576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98257efd-bc57-458e-91c4-21be930f227a admin admin] PATH_INFO: `/v3/projects/f7e238fd6e744925b3ddba810e541393/users/2092124575d64ec4b2a595bb5f0ca35a/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:35.847599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98257efd-bc57-458e-91c4-21be930f227a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e238fd6e744925b3ddba810e541393, user_id=2092124575d64ec4b2a595bb5f0ca35a, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:35.849228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98257efd-bc57-458e-91c4-21be930f227a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:35.864688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 995/1993] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:35 2022] PUT /identity/v3/projects/f7e238fd6e744925b3ddba810e541393/users/2092124575d64ec4b2a595bb5f0ca35a/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:35.874807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f195a0-b78a-4d5e-98ad-e5da10065ac1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:35.875146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f195a0-b78a-4d5e-98ad-e5da10065ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:35.875428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f195a0-b78a-4d5e-98ad-e5da10065ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:35.910266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-62f195a0-b78a-4d5e-98ad-e5da10065ac1 None None] MFA Rules not processed for user `2092124575d64ec4b2a595bb5f0ca35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:35.911708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-62f195a0-b78a-4d5e-98ad-e5da10065ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:35.916408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:35.991420 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 999/1994] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:36.002480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98257efd-bc57-458e-91c4-21be930f227a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:36.004857 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-98257efd-bc57-458e-91c4-21be930f227a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:36.105878 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98257efd-bc57-458e-91c4-21be930f227a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:36.112973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa561b-8ff7-4e8e-b859-9e9f0b02de6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:36.113804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa561b-8ff7-4e8e-b859-9e9f0b02de6d service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:36.115297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa561b-8ff7-4e8e-b859-9e9f0b02de6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:36.115584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa561b-8ff7-4e8e-b859-9e9f0b02de6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:36.132006 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa561b-8ff7-4e8e-b859-9e9f0b02de6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:36.133658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa561b-8ff7-4e8e-b859-9e9f0b02de6d service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:36.144843 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 996/1995] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:36.165598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f195a0-b78a-4d5e-98ad-e5da10065ac1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:36.254043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa561b-8ff7-4e8e-b859-9e9f0b02de6d service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:36.255511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-28aa561b-8ff7-4e8e-b859-9e9f0b02de6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:36.300174 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f195a0-b78a-4d5e-98ad-e5da10065ac1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:36.306419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31838bb3-17b9-4348-85df-26fa3007706d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:36.307329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31838bb3-17b9-4348-85df-26fa3007706d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:36.307789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31838bb3-17b9-4348-85df-26fa3007706d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:36.308232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31838bb3-17b9-4348-85df-26fa3007706d admin admin] PATH_INFO: `/v3/users/2092124575d64ec4b2a595bb5f0ca35a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:36.309633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31838bb3-17b9-4348-85df-26fa3007706d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2092124575d64ec4b2a595bb5f0ca35a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:36.312098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31838bb3-17b9-4348-85df-26fa3007706d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:36.348786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:36.373664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa561b-8ff7-4e8e-b859-9e9f0b02de6d service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:36.378833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078132d6-66e9-46e2-a9bd-323fc2effb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:36.379700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078132d6-66e9-46e2-a9bd-323fc2effb01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:36.380095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078132d6-66e9-46e2-a9bd-323fc2effb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:36.380407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078132d6-66e9-46e2-a9bd-323fc2effb01 admin admin] PATH_INFO: `/v3/projects/b93ff6b02eae473db54c9fa3d5633248` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:36.382079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078132d6-66e9-46e2-a9bd-323fc2effb01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b93ff6b02eae473db54c9fa3d5633248)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:36.384375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078132d6-66e9-46e2-a9bd-323fc2effb01 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:36.391698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-31838bb3-17b9-4348-85df-26fa3007706d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2092124575d64ec4b2a595bb5f0ca35a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:36.392431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-31838bb3-17b9-4348-85df-26fa3007706d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2092124575d64ec4b2a595bb5f0ca35a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:36.403076 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-31838bb3-17b9-4348-85df-26fa3007706d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2092124575d64ec4b2a595bb5f0ca35a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:36.419343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-31838bb3-17b9-4348-85df-26fa3007706d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2092124575d64ec4b2a595bb5f0ca35a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:36.419998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-31838bb3-17b9-4348-85df-26fa3007706d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2092124575d64ec4b2a595bb5f0ca35a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:36.437598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1000/1996] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:36 2022] DELETE /identity/v3/users/2092124575d64ec4b2a595bb5f0ca35a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:36.442232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f525e17-70c9-458b-959a-cfcf1b6604a7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:36.442684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f525e17-70c9-458b-959a-cfcf1b6604a7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:36.443001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f525e17-70c9-458b-959a-cfcf1b6604a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:36.457609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-078132d6-66e9-46e2-a9bd-323fc2effb01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b93ff6b02eae473db54c9fa3d5633248', 'request_id': 'req-078132d6-66e9-46e2-a9bd-323fc2effb01'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:36.468066 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1f525e17-70c9-458b-959a-cfcf1b6604a7 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:36.469747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1f525e17-70c9-458b-959a-cfcf1b6604a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:36.472131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-078132d6-66e9-46e2-a9bd-323fc2effb01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b93ff6b02eae473db54c9fa3d5633248', 'request_id': 'req-078132d6-66e9-46e2-a9bd-323fc2effb01'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:36.482699 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-078132d6-66e9-46e2-a9bd-323fc2effb01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b93ff6b02eae473db54c9fa3d5633248', 'request_id': 'req-078132d6-66e9-46e2-a9bd-323fc2effb01'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:36.492738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:36.500595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-078132d6-66e9-46e2-a9bd-323fc2effb01 admin admin] The token cache is being invalidate because project b93ff6b02eae473db54c9fa3d5633248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:36.501213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-078132d6-66e9-46e2-a9bd-323fc2effb01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:36.502104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-078132d6-66e9-46e2-a9bd-323fc2effb01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:36.505308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 997/1997] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:36 2022] DELETE /identity/v3/projects/b93ff6b02eae473db54c9fa3d5633248 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:15:36.515156 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9791d5-56df-4b3b-8494-038a05df7933 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:36.516850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9791d5-56df-4b3b-8494-038a05df7933 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:36.516850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9791d5-56df-4b3b-8494-038a05df7933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:36.542544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fb9791d5-56df-4b3b-8494-038a05df7933 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:36.544613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fb9791d5-56df-4b3b-8494-038a05df7933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:36.563308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1001/1998] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:36.574192 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f525e17-70c9-458b-959a-cfcf1b6604a7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:36.575634 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1f525e17-70c9-458b-959a-cfcf1b6604a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:36.640807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 998/1999] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:36.649580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89b61c-5ccb-4e10-b1b9-cb40efe86d2c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:36.650108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89b61c-5ccb-4e10-b1b9-cb40efe86d2c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:36.650533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89b61c-5ccb-4e10-b1b9-cb40efe86d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:36.675658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-be89b61c-5ccb-4e10-b1b9-cb40efe86d2c None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:36.676943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-be89b61c-5ccb-4e10-b1b9-cb40efe86d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:36.679934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f525e17-70c9-458b-959a-cfcf1b6604a7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:36.685453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bf374-9b08-4ae5-b4ff-cc78de1c22c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:36.686438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bf374-9b08-4ae5-b4ff-cc78de1c22c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:36.686574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bf374-9b08-4ae5-b4ff-cc78de1c22c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:36.686945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bf374-9b08-4ae5-b4ff-cc78de1c22c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:36.692814 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 999/2000] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:36.697080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89b61c-5ccb-4e10-b1b9-cb40efe86d2c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:36.704432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bf374-9b08-4ae5-b4ff-cc78de1c22c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:36.707323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bf374-9b08-4ae5-b4ff-cc78de1c22c9 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:36.707452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89b61c-5ccb-4e10-b1b9-cb40efe86d2c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:36.712443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bd81a-754e-42fc-ba6f-f278103be860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:36.713134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bd81a-754e-42fc-ba6f-f278103be860 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:36.713414 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bd81a-754e-42fc-ba6f-f278103be860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:36.713682 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bd81a-754e-42fc-ba6f-f278103be860 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:36.714425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bd81a-754e-42fc-ba6f-f278103be860 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:15:36.714618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bd81a-754e-42fc-ba6f-f278103be860 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:36.717957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bd81a-754e-42fc-ba6f-f278103be860 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:36.734509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1000/2001] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:15:36.735996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1002/2002] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:36.739105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bd81a-754e-42fc-ba6f-f278103be860 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:36.755418 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bd81a-754e-42fc-ba6f-f278103be860 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:36.763260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28145d-8a05-4522-bc31-81854ddb8776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:36.765324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28145d-8a05-4522-bc31-81854ddb8776 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:36.765559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28145d-8a05-4522-bc31-81854ddb8776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:36.765838 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28145d-8a05-4522-bc31-81854ddb8776 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:36.768739 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28145d-8a05-4522-bc31-81854ddb8776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:36.772111 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28145d-8a05-4522-bc31-81854ddb8776 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:36.846021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1001/2003] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:36 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:15:36.848900 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bf374-9b08-4ae5-b4ff-cc78de1c22c9 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:36.869425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bf374-9b08-4ae5-b4ff-cc78de1c22c9 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:36.930846 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28145d-8a05-4522-bc31-81854ddb8776 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:36.932545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cf28145d-8a05-4522-bc31-81854ddb8776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:36.971170 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:36.979298 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29885efb-7538-4179-9946-40bd4ae168fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:36.980007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29885efb-7538-4179-9946-40bd4ae168fa admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:36.980575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29885efb-7538-4179-9946-40bd4ae168fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:36.980959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29885efb-7538-4179-9946-40bd4ae168fa admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:36.981652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29885efb-7538-4179-9946-40bd4ae168fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:36.984360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29885efb-7538-4179-9946-40bd4ae168fa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:37.009581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28145d-8a05-4522-bc31-81854ddb8776 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:37.015793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eab798-2f3f-443b-b2e0-d99f9f2c356a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:37.016464 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eab798-2f3f-443b-b2e0-d99f9f2c356a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:37.016805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eab798-2f3f-443b-b2e0-d99f9f2c356a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:37.017110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eab798-2f3f-443b-b2e0-d99f9f2c356a admin admin] PATH_INFO: `/v3/users/3f193ec725a54b08844236fd695d236a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:37.018445 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eab798-2f3f-443b-b2e0-d99f9f2c356a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f193ec725a54b08844236fd695d236a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:37.020476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eab798-2f3f-443b-b2e0-d99f9f2c356a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:37.027008 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1003/2004] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:36 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:37.031363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29885efb-7538-4179-9946-40bd4ae168fa admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:37.055445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29885efb-7538-4179-9946-40bd4ae168fa admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:37.081888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de424cb-e142-459a-a8d8-1b6e5eef5541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:37.082766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de424cb-e142-459a-a8d8-1b6e5eef5541 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:37.083045 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de424cb-e142-459a-a8d8-1b6e5eef5541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:37.083331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de424cb-e142-459a-a8d8-1b6e5eef5541 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:37.083996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de424cb-e142-459a-a8d8-1b6e5eef5541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:37.085712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de424cb-e142-459a-a8d8-1b6e5eef5541 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:37.100291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1004/2005] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:37.101584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de424cb-e142-459a-a8d8-1b6e5eef5541 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:37.112993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de424cb-e142-459a-a8d8-1b6e5eef5541 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:37.113571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-50eab798-2f3f-443b-b2e0-d99f9f2c356a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f193ec725a54b08844236fd695d236a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:37.114918 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-50eab798-2f3f-443b-b2e0-d99f9f2c356a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f193ec725a54b08844236fd695d236a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:37.118624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94261f91-165a-40e5-a8c6-3fbca5409e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:37.119482 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94261f91-165a-40e5-a8c6-3fbca5409e04 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:37.119604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94261f91-165a-40e5-a8c6-3fbca5409e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:37.119969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94261f91-165a-40e5-a8c6-3fbca5409e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:37.120747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94261f91-165a-40e5-a8c6-3fbca5409e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:37.122705 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94261f91-165a-40e5-a8c6-3fbca5409e04 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:37.128407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-50eab798-2f3f-443b-b2e0-d99f9f2c356a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f193ec725a54b08844236fd695d236a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:37.135252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1005/2006] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:37.139381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94261f91-165a-40e5-a8c6-3fbca5409e04 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:37.141444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-94261f91-165a-40e5-a8c6-3fbca5409e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:37.151181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-50eab798-2f3f-443b-b2e0-d99f9f2c356a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f193ec725a54b08844236fd695d236a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:37.152986 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-50eab798-2f3f-443b-b2e0-d99f9f2c356a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f193ec725a54b08844236fd695d236a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:37.158312 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94261f91-165a-40e5-a8c6-3fbca5409e04 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:37.165798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fecae-eeb2-4b0d-9819-9f15b118fa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:37.166564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fecae-eeb2-4b0d-9819-9f15b118fa29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:37.167203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fecae-eeb2-4b0d-9819-9f15b118fa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:37.167203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fecae-eeb2-4b0d-9819-9f15b118fa29 admin admin] PATH_INFO: `/v3/projects/04f17cdc53b3464da70821f8926086a6/users/b784fd0a96994a04894fcc806c3f9968/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:37.180070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1002/2007] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:36 2022] DELETE /identity/v3/users/3f193ec725a54b08844236fd695d236a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:15:37.184974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476640d-e9ac-40aa-a0fe-f65c50f10fef None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:37.186223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476640d-e9ac-40aa-a0fe-f65c50f10fef None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:37.188297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476640d-e9ac-40aa-a0fe-f65c50f10fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:37.194511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fecae-eeb2-4b0d-9819-9f15b118fa29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f17cdc53b3464da70821f8926086a6, user_id=b784fd0a96994a04894fcc806c3f9968, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:37.198502 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fecae-eeb2-4b0d-9819-9f15b118fa29 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:37.224029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1006/2008] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:37 2022] PUT /identity/v3/projects/04f17cdc53b3464da70821f8926086a6/users/b784fd0a96994a04894fcc806c3f9968/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:37.233589 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d476640d-e9ac-40aa-a0fe-f65c50f10fef None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:37.235007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d476640d-e9ac-40aa-a0fe-f65c50f10fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:37.236233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468d57e-61fb-4580-99e8-141c5ebeae78 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:37.236692 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468d57e-61fb-4580-99e8-141c5ebeae78 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:37.237136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468d57e-61fb-4580-99e8-141c5ebeae78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:37.267164 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:37.292183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3468d57e-61fb-4580-99e8-141c5ebeae78 None None] MFA Rules not processed for user `b784fd0a96994a04894fcc806c3f9968`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:37.293815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3468d57e-61fb-4580-99e8-141c5ebeae78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:37.365112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1003/2009] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:37.375796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476640d-e9ac-40aa-a0fe-f65c50f10fef None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:37.377755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d476640d-e9ac-40aa-a0fe-f65c50f10fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:37.417484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1007/2010] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:37.430603 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468d57e-61fb-4580-99e8-141c5ebeae78 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:37.433685 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3468d57e-61fb-4580-99e8-141c5ebeae78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:37.472395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476640d-e9ac-40aa-a0fe-f65c50f10fef None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:37.477148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc25d0-93d1-48bc-b441-88e64bc20eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:37.478968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc25d0-93d1-48bc-b441-88e64bc20eef service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:37.480341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc25d0-93d1-48bc-b441-88e64bc20eef service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:37.480987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc25d0-93d1-48bc-b441-88e64bc20eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:37.497926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:37.502531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc25d0-93d1-48bc-b441-88e64bc20eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:37.506417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc25d0-93d1-48bc-b441-88e64bc20eef service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:37.534032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1004/2011] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:37.538964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468d57e-61fb-4580-99e8-141c5ebeae78 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:37.544685 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350880a4-cb53-472c-86b0-b9dc7d90b0f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:37.546659 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350880a4-cb53-472c-86b0-b9dc7d90b0f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:37.546940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350880a4-cb53-472c-86b0-b9dc7d90b0f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:37.547214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350880a4-cb53-472c-86b0-b9dc7d90b0f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:37.562095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350880a4-cb53-472c-86b0-b9dc7d90b0f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:37.564124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350880a4-cb53-472c-86b0-b9dc7d90b0f3 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:37.578582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1008/2012] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:37.809812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc25d0-93d1-48bc-b441-88e64bc20eef service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:37.811151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5ddc25d0-93d1-48bc-b441-88e64bc20eef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:37.841167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc25d0-93d1-48bc-b441-88e64bc20eef service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:37.887443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28233661-b4c3-45af-89cf-b7c72b9a255b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:37.888149 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28233661-b4c3-45af-89cf-b7c72b9a255b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:37.888427 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28233661-b4c3-45af-89cf-b7c72b9a255b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:37.888713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28233661-b4c3-45af-89cf-b7c72b9a255b admin admin] PATH_INFO: `/v3/projects/f7e238fd6e744925b3ddba810e541393` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:37.897948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28233661-b4c3-45af-89cf-b7c72b9a255b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e238fd6e744925b3ddba810e541393)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:37.900863 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28233661-b4c3-45af-89cf-b7c72b9a255b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:38.000149 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-28233661-b4c3-45af-89cf-b7c72b9a255b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e238fd6e744925b3ddba810e541393', 'request_id': 'req-28233661-b4c3-45af-89cf-b7c72b9a255b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.001654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350880a4-cb53-472c-86b0-b9dc7d90b0f3 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:38.007433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-350880a4-cb53-472c-86b0-b9dc7d90b0f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:38.010911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-28233661-b4c3-45af-89cf-b7c72b9a255b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e238fd6e744925b3ddba810e541393', 'request_id': 'req-28233661-b4c3-45af-89cf-b7c72b9a255b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.019793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-28233661-b4c3-45af-89cf-b7c72b9a255b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e238fd6e744925b3ddba810e541393', 'request_id': 'req-28233661-b4c3-45af-89cf-b7c72b9a255b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.036100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-28233661-b4c3-45af-89cf-b7c72b9a255b admin admin] The token cache is being invalidate because project f7e238fd6e744925b3ddba810e541393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:38.036413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-28233661-b4c3-45af-89cf-b7c72b9a255b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.037192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-28233661-b4c3-45af-89cf-b7c72b9a255b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.044751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1005/2013] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:37 2022] DELETE /identity/v3/projects/f7e238fd6e744925b3ddba810e541393 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:38.046154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b4221-2521-4bb3-8d72-4540cfacb0d0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:38.046545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b4221-2521-4bb3-8d72-4540cfacb0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:38.046887 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b4221-2521-4bb3-8d72-4540cfacb0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:38.061111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350880a4-cb53-472c-86b0-b9dc7d90b0f3 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:38.076575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b83b4221-2521-4bb3-8d72-4540cfacb0d0 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:38.078965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b83b4221-2521-4bb3-8d72-4540cfacb0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:38.213882 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1006/2014] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:38.215542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac51f3-22fd-420b-b01f-32fc13a275e2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:38.215834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac51f3-22fd-420b-b01f-32fc13a275e2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:38.216122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac51f3-22fd-420b-b01f-32fc13a275e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:38.251296 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2cac51f3-22fd-420b-b01f-32fc13a275e2 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:38.252732 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2cac51f3-22fd-420b-b01f-32fc13a275e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:38.253957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:38.255234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:38.255696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:38.256107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] PATH_INFO: `/v3/projects/87321fcd93f84f5e9e1c97cd23118b1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:38.257551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] RBAC: Authorizing `identity:delete_project(project_id=87321fcd93f84f5e9e1c97cd23118b1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:38.260095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:38.269101 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1007/2015] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:38.274886 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac51f3-22fd-420b-b01f-32fc13a275e2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:38.285581 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac51f3-22fd-420b-b01f-32fc13a275e2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:38.290946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b4896-b6e7-442c-b482-68b615668498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:38.291709 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b4896-b6e7-442c-b482-68b615668498 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:38.292097 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b4896-b6e7-442c-b482-68b615668498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:38.292380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b4896-b6e7-442c-b482-68b615668498 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:38.293282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b4896-b6e7-442c-b482-68b615668498 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:15:38.293844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b4896-b6e7-442c-b482-68b615668498 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:38.295614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b4896-b6e7-442c-b482-68b615668498 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:38.335529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1008/2016] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:15:38.336599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b4896-b6e7-442c-b482-68b615668498 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:38.358506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b4896-b6e7-442c-b482-68b615668498 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:38.375565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:38.385308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87321fcd93f84f5e9e1c97cd23118b1e', 'request_id': 'req-b70d5746-a919-4423-8205-9fe45a28329c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.397929 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87321fcd93f84f5e9e1c97cd23118b1e', 'request_id': 'req-b70d5746-a919-4423-8205-9fe45a28329c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.412168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87321fcd93f84f5e9e1c97cd23118b1e', 'request_id': 'req-b70d5746-a919-4423-8205-9fe45a28329c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.431284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] The token cache is being invalidate because project 87321fcd93f84f5e9e1c97cd23118b1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:38.431717 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.432541 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.434184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1009/2017] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:38 2022] DELETE /identity/v3/projects/87321fcd93f84f5e9e1c97cd23118b1e => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:38.438208 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:38.439779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:38.483457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9a02f-20c5-4051-a5d5-8d0b032abed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:38.485429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9a02f-20c5-4051-a5d5-8d0b032abed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:38.486080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9a02f-20c5-4051-a5d5-8d0b032abed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:38.486603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9a02f-20c5-4051-a5d5-8d0b032abed1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:38.487536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9a02f-20c5-4051-a5d5-8d0b032abed1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:38.490053 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9a02f-20c5-4051-a5d5-8d0b032abed1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:38.497117 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:38.536326 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1009/2018] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:38 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:38.540124 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9a02f-20c5-4051-a5d5-8d0b032abed1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:38.542258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bde9a02f-20c5-4051-a5d5-8d0b032abed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:38.545785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d5746-a919-4423-8205-9fe45a28329c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:38.643967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9a02f-20c5-4051-a5d5-8d0b032abed1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:38.645136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c190a-a439-4cb4-bf6d-63435426fb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:38.645844 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c190a-a439-4cb4-bf6d-63435426fb54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:38.646141 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c190a-a439-4cb4-bf6d-63435426fb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:38.646392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c190a-a439-4cb4-bf6d-63435426fb54 admin admin] PATH_INFO: `/v3/users/e3e3f325e24a475ca817c737a65ea553` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:38.647812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c190a-a439-4cb4-bf6d-63435426fb54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3e3f325e24a475ca817c737a65ea553)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:38.651379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001353e7-322d-486f-865d-f4c24cda44b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:38.652197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001353e7-322d-486f-865d-f4c24cda44b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:38.652612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001353e7-322d-486f-865d-f4c24cda44b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:38.653135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001353e7-322d-486f-865d-f4c24cda44b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:38.654066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001353e7-322d-486f-865d-f4c24cda44b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:38.656186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c190a-a439-4cb4-bf6d-63435426fb54 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:38.656331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001353e7-322d-486f-865d-f4c24cda44b4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:38.686374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1010/2019] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:38 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:15:38.690101 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001353e7-322d-486f-865d-f4c24cda44b4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:38.708871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001353e7-322d-486f-865d-f4c24cda44b4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:38.713955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a588-e1e0-4833-80f8-c2e094c5b498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:38.714593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a588-e1e0-4833-80f8-c2e094c5b498 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:38.714871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a588-e1e0-4833-80f8-c2e094c5b498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:38.715142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a588-e1e0-4833-80f8-c2e094c5b498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:38.715825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980a588-e1e0-4833-80f8-c2e094c5b498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:38.726246 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980a588-e1e0-4833-80f8-c2e094c5b498 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:38.739477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a61c190a-a439-4cb4-bf6d-63435426fb54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3e3f325e24a475ca817c737a65ea553'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.740410 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a61c190a-a439-4cb4-bf6d-63435426fb54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3e3f325e24a475ca817c737a65ea553'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.751028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1011/2020] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:38.755486 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a61c190a-a439-4cb4-bf6d-63435426fb54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3e3f325e24a475ca817c737a65ea553'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.763192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a588-e1e0-4833-80f8-c2e094c5b498 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:38.767380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5980a588-e1e0-4833-80f8-c2e094c5b498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:38.779616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a61c190a-a439-4cb4-bf6d-63435426fb54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3e3f325e24a475ca817c737a65ea553'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.780411 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a61c190a-a439-4cb4-bf6d-63435426fb54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3e3f325e24a475ca817c737a65ea553'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:38.792037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a588-e1e0-4833-80f8-c2e094c5b498 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:38.794778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:38.801327 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1010/2021] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:38 2022] DELETE /identity/v3/users/e3e3f325e24a475ca817c737a65ea553 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:38.807034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965f2a2-1908-45aa-86bb-016da3bd35a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:38.807549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965f2a2-1908-45aa-86bb-016da3bd35a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:38.808015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965f2a2-1908-45aa-86bb-016da3bd35a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:38.808124 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965f2a2-1908-45aa-86bb-016da3bd35a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:38.809075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965f2a2-1908-45aa-86bb-016da3bd35a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:38.810720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965f2a2-1908-45aa-86bb-016da3bd35a3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:38.823599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1012/2022] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:38.828341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c190a-a439-4cb4-bf6d-63435426fb54 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:38.908358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c190a-a439-4cb4-bf6d-63435426fb54 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:38.915241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0ef86-6188-436f-b557-36f31aa1dc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:38.915986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0ef86-6188-436f-b557-36f31aa1dc13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:38.916286 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0ef86-6188-436f-b557-36f31aa1dc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:38.916561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0ef86-6188-436f-b557-36f31aa1dc13 admin admin] PATH_INFO: `/v3/projects/76a0436f87634bc9bbe55b249305503e/users/a2ce2bb2f2ca449884bab2cf432d20de/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:38.937233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0ef86-6188-436f-b557-36f31aa1dc13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76a0436f87634bc9bbe55b249305503e, user_id=a2ce2bb2f2ca449884bab2cf432d20de, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:38.940203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0ef86-6188-436f-b557-36f31aa1dc13 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:38.965073 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1011/2023] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:38 2022] PUT /identity/v3/projects/76a0436f87634bc9bbe55b249305503e/users/a2ce2bb2f2ca449884bab2cf432d20de/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:38.976393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb7f9f-e65d-4cf2-8ef7-a883d92e33f1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:38.977121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb7f9f-e65d-4cf2-8ef7-a883d92e33f1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:38.977224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb7f9f-e65d-4cf2-8ef7-a883d92e33f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:39.022364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ddeb7f9f-e65d-4cf2-8ef7-a883d92e33f1 None None] MFA Rules not processed for user `a2ce2bb2f2ca449884bab2cf432d20de`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:39.024027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ddeb7f9f-e65d-4cf2-8ef7-a883d92e33f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:39.136262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1013/2024] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:39.141438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0ef86-6188-436f-b557-36f31aa1dc13 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:39.152410 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb7f9f-e65d-4cf2-8ef7-a883d92e33f1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:39.156756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ddeb7f9f-e65d-4cf2-8ef7-a883d92e33f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:39.288013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:39.317406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb7f9f-e65d-4cf2-8ef7-a883d92e33f1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:39.318006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0ef86-6188-436f-b557-36f31aa1dc13 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:39.322993 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:39.323956 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:39.324282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:39.324488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384289d2-e22f-4e15-a29a-e5b20fa9f801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:39.324589 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] PATH_INFO: `/v3/projects/f14ae0e55da249dba89961f0339fd655` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:39.325402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384289d2-e22f-4e15-a29a-e5b20fa9f801 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:39.325692 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384289d2-e22f-4e15-a29a-e5b20fa9f801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:39.326160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384289d2-e22f-4e15-a29a-e5b20fa9f801 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:39.326342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f14ae0e55da249dba89961f0339fd655)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:39.327727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384289d2-e22f-4e15-a29a-e5b20fa9f801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:39.328083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:39.331001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384289d2-e22f-4e15-a29a-e5b20fa9f801 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:39.368761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1012/2025] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:39 2022] POST /identity/v3/projects => generated 383 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:39.372791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384289d2-e22f-4e15-a29a-e5b20fa9f801 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:39.405384 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f14ae0e55da249dba89961f0339fd655', 'request_id': 'req-2b88461d-6550-48fd-9b83-19fc341dbd3d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:39.435596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f14ae0e55da249dba89961f0339fd655', 'request_id': 'req-2b88461d-6550-48fd-9b83-19fc341dbd3d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:39.443865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f14ae0e55da249dba89961f0339fd655', 'request_id': 'req-2b88461d-6550-48fd-9b83-19fc341dbd3d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:39.471612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] The token cache is being invalidate because project f14ae0e55da249dba89961f0339fd655 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:39.472030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:39.472872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:39.475051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1014/2026] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:39 2022] DELETE /identity/v3/projects/f14ae0e55da249dba89961f0339fd655 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:39.479079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:39.480461 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:39.494185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384289d2-e22f-4e15-a29a-e5b20fa9f801 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:39.507219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:39.552191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88461d-6550-48fd-9b83-19fc341dbd3d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:39.556964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c784e-c4fc-4b3f-b356-7d8547b8de72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:39.557600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c784e-c4fc-4b3f-b356-7d8547b8de72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:39.557878 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c784e-c4fc-4b3f-b356-7d8547b8de72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:39.558145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c784e-c4fc-4b3f-b356-7d8547b8de72 admin admin] PATH_INFO: `/v3/users/ddba77508406430f920ab8e54429b6a2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:39.558980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59e028-eab1-4904-9987-d69cf68f2b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:39.559520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c784e-c4fc-4b3f-b356-7d8547b8de72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddba77508406430f920ab8e54429b6a2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:39.559641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59e028-eab1-4904-9987-d69cf68f2b8b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:39.559892 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59e028-eab1-4904-9987-d69cf68f2b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:39.560165 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59e028-eab1-4904-9987-d69cf68f2b8b admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:39.560835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59e028-eab1-4904-9987-d69cf68f2b8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:39.562003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c784e-c4fc-4b3f-b356-7d8547b8de72 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:39.562885 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59e028-eab1-4904-9987-d69cf68f2b8b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:39.586738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1013/2027] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:39 2022] POST /identity/v3/users => generated 292 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:39.591068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59e028-eab1-4904-9987-d69cf68f2b8b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:39.593033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4a59e028-eab1-4904-9987-d69cf68f2b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:39.605798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59e028-eab1-4904-9987-d69cf68f2b8b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:39.611215 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd3370-e097-4350-808a-be5c70438285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:39.611934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd3370-e097-4350-808a-be5c70438285 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:39.612285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd3370-e097-4350-808a-be5c70438285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:39.613015 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd3370-e097-4350-808a-be5c70438285 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:39.613517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd3370-e097-4350-808a-be5c70438285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:39.615419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd3370-e097-4350-808a-be5c70438285 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:39.623633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-282c784e-c4fc-4b3f-b356-7d8547b8de72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddba77508406430f920ab8e54429b6a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:39.624315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-282c784e-c4fc-4b3f-b356-7d8547b8de72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddba77508406430f920ab8e54429b6a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:39.627153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1014/2028] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:39.631139 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd3370-e097-4350-808a-be5c70438285 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:39.631909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-282c784e-c4fc-4b3f-b356-7d8547b8de72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddba77508406430f920ab8e54429b6a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:39.632847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c2dd3370-e097-4350-808a-be5c70438285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:39.644077 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-282c784e-c4fc-4b3f-b356-7d8547b8de72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddba77508406430f920ab8e54429b6a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:39.644953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-282c784e-c4fc-4b3f-b356-7d8547b8de72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddba77508406430f920ab8e54429b6a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:39.645091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd3370-e097-4350-808a-be5c70438285 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:39.646679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:39.651177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0800662-3242-4dbf-a670-79b8c39eafaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:39.651804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0800662-3242-4dbf-a670-79b8c39eafaa admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:39.652068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0800662-3242-4dbf-a670-79b8c39eafaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:39.652350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0800662-3242-4dbf-a670-79b8c39eafaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:39.653010 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0800662-3242-4dbf-a670-79b8c39eafaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:39.654660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0800662-3242-4dbf-a670-79b8c39eafaa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:39.665528 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1015/2029] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:39 2022] DELETE /identity/v3/users/ddba77508406430f920ab8e54429b6a2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:15:39.667774 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1015/2030] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:15:39.672826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c784e-c4fc-4b3f-b356-7d8547b8de72 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:39.748310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c784e-c4fc-4b3f-b356-7d8547b8de72 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:39.753816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95dee1-cf6c-426d-820b-4a788bb55967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:39.754488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95dee1-cf6c-426d-820b-4a788bb55967 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:39.754761 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95dee1-cf6c-426d-820b-4a788bb55967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:39.755030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95dee1-cf6c-426d-820b-4a788bb55967 admin admin] PATH_INFO: `/v3/projects/254fef49f402453a9e51dfbe60eb740f/users/c57bdc898edc4fcf8cd428044de97973/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:39.771500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95dee1-cf6c-426d-820b-4a788bb55967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=254fef49f402453a9e51dfbe60eb740f, user_id=c57bdc898edc4fcf8cd428044de97973, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:39.774637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95dee1-cf6c-426d-820b-4a788bb55967 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:39.788554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1016/2031] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:39 2022] PUT /identity/v3/projects/254fef49f402453a9e51dfbe60eb740f/users/c57bdc898edc4fcf8cd428044de97973/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:39.798901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483e40e-0a63-40a1-b290-dd54ba74aa51 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:39.799557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483e40e-0a63-40a1-b290-dd54ba74aa51 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:39.799827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483e40e-0a63-40a1-b290-dd54ba74aa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:39.833978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b483e40e-0a63-40a1-b290-dd54ba74aa51 None None] MFA Rules not processed for user `c57bdc898edc4fcf8cd428044de97973`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:39.835473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b483e40e-0a63-40a1-b290-dd54ba74aa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:39.914217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1016/2032] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:39.919234 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95dee1-cf6c-426d-820b-4a788bb55967 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:39.965510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483e40e-0a63-40a1-b290-dd54ba74aa51 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:39.967048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b483e40e-0a63-40a1-b290-dd54ba74aa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:39.994782 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95dee1-cf6c-426d-820b-4a788bb55967 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:39.999776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7a292-4cca-453c-8350-a709c5453f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.000390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7a292-4cca-453c-8350-a709c5453f43 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.000683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7a292-4cca-453c-8350-a709c5453f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.000970 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7a292-4cca-453c-8350-a709c5453f43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.001642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7a292-4cca-453c-8350-a709c5453f43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.003642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7a292-4cca-453c-8350-a709c5453f43 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.032484 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1017/2033] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:39 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:40.036749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7a292-4cca-453c-8350-a709c5453f43 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:40.044567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483e40e-0a63-40a1-b290-dd54ba74aa51 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:40.061956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:40.104710 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7a292-4cca-453c-8350-a709c5453f43 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:40.106587 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209f7c4-199b-435f-b026-2c52eb41c946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.107236 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209f7c4-199b-435f-b026-2c52eb41c946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.107513 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209f7c4-199b-435f-b026-2c52eb41c946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.107837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209f7c4-199b-435f-b026-2c52eb41c946 admin admin] PATH_INFO: `/v3/projects/9ec4174aacf64c548e53ccde9d24cd06` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.108931 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209f7c4-199b-435f-b026-2c52eb41c946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec4174aacf64c548e53ccde9d24cd06)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.109411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250007d-b79d-44ac-9b72-24fe79b09165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.110023 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250007d-b79d-44ac-9b72-24fe79b09165 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.110305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250007d-b79d-44ac-9b72-24fe79b09165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.110578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250007d-b79d-44ac-9b72-24fe79b09165 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.110855 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209f7c4-199b-435f-b026-2c52eb41c946 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.111242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250007d-b79d-44ac-9b72-24fe79b09165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.113386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250007d-b79d-44ac-9b72-24fe79b09165 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.122021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:40.138717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1018/2034] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:40 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:15:40.142940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250007d-b79d-44ac-9b72-24fe79b09165 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:40.152783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250007d-b79d-44ac-9b72-24fe79b09165 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:40.162408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8209f7c4-199b-435f-b026-2c52eb41c946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec4174aacf64c548e53ccde9d24cd06', 'request_id': 'req-8209f7c4-199b-435f-b026-2c52eb41c946'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:40.168676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8209f7c4-199b-435f-b026-2c52eb41c946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec4174aacf64c548e53ccde9d24cd06', 'request_id': 'req-8209f7c4-199b-435f-b026-2c52eb41c946'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:40.174772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8209f7c4-199b-435f-b026-2c52eb41c946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec4174aacf64c548e53ccde9d24cd06', 'request_id': 'req-8209f7c4-199b-435f-b026-2c52eb41c946'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:40.187150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8209f7c4-199b-435f-b026-2c52eb41c946 admin admin] The token cache is being invalidate because project 9ec4174aacf64c548e53ccde9d24cd06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:40.187456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8209f7c4-199b-435f-b026-2c52eb41c946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:40.188104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8209f7c4-199b-435f-b026-2c52eb41c946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:40.189709 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1017/2035] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:39 2022] DELETE /identity/v3/projects/9ec4174aacf64c548e53ccde9d24cd06 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:40.196140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5fe69-949e-4568-80d3-158daba6def1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.196418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5fe69-949e-4568-80d3-158daba6def1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.196737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5fe69-949e-4568-80d3-158daba6def1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.219374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b246b-4926-44b6-b5cd-90a42e1b7f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.219823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-15c5fe69-949e-4568-80d3-158daba6def1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:40.219997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b246b-4926-44b6-b5cd-90a42e1b7f29 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.220275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b246b-4926-44b6-b5cd-90a42e1b7f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.220554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b246b-4926-44b6-b5cd-90a42e1b7f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.221125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-15c5fe69-949e-4568-80d3-158daba6def1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:40.221244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b246b-4926-44b6-b5cd-90a42e1b7f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.224426 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b246b-4926-44b6-b5cd-90a42e1b7f29 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.231345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1018/2036] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:40.234069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1019/2037] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:15:40.238327 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5fe69-949e-4568-80d3-158daba6def1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:40.239887 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-15c5fe69-949e-4568-80d3-158daba6def1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:40.241079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064bcbb-0c6c-4b6d-8a5b-a31b4b2408bb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.241633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064bcbb-0c6c-4b6d-8a5b-a31b4b2408bb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.242017 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064bcbb-0c6c-4b6d-8a5b-a31b4b2408bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.253467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5fe69-949e-4568-80d3-158daba6def1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:40.258173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c41bca-b19b-4de9-9b90-1559e27f6822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.258869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c41bca-b19b-4de9-9b90-1559e27f6822 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.259176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c41bca-b19b-4de9-9b90-1559e27f6822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.259418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c41bca-b19b-4de9-9b90-1559e27f6822 admin admin] PATH_INFO: `/v3/projects/93630b505e70436dbbb5fb3337b974c5/users/98aa7cd13484414f8f32104bd91072e7/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.264053 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5064bcbb-0c6c-4b6d-8a5b-a31b4b2408bb None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:40.265277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5064bcbb-0c6c-4b6d-8a5b-a31b4b2408bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:40.273795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1020/2038] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:40.277516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c41bca-b19b-4de9-9b90-1559e27f6822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93630b505e70436dbbb5fb3337b974c5, user_id=98aa7cd13484414f8f32104bd91072e7, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.278215 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064bcbb-0c6c-4b6d-8a5b-a31b4b2408bb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:40.280644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c41bca-b19b-4de9-9b90-1559e27f6822 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.289649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064bcbb-0c6c-4b6d-8a5b-a31b4b2408bb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:40.295587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39fca6-08bb-4d5d-80f8-82b8be4f7135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.296327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39fca6-08bb-4d5d-80f8-82b8be4f7135 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.296615 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39fca6-08bb-4d5d-80f8-82b8be4f7135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.296949 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39fca6-08bb-4d5d-80f8-82b8be4f7135 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.297702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39fca6-08bb-4d5d-80f8-82b8be4f7135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:15:40.298060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39fca6-08bb-4d5d-80f8-82b8be4f7135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.300084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39fca6-08bb-4d5d-80f8-82b8be4f7135 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.308511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1019/2039] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:40 2022] PUT /identity/v3/projects/93630b505e70436dbbb5fb3337b974c5/users/98aa7cd13484414f8f32104bd91072e7/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:40.309600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c41bca-b19b-4de9-9b90-1559e27f6822 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:40.315254 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1021/2040] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:15:40.320719 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39fca6-08bb-4d5d-80f8-82b8be4f7135 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:40.335140 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39fca6-08bb-4d5d-80f8-82b8be4f7135 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:40.391653 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c41bca-b19b-4de9-9b90-1559e27f6822 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:40.396834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa3101-3f36-4c19-ab80-384642433dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.397572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa3101-3f36-4c19-ab80-384642433dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.398052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa3101-3f36-4c19-ab80-384642433dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.398150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa3101-3f36-4c19-ab80-384642433dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.398278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85503aaf-d447-4327-8a0f-08232ce31e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.398732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa3101-3f36-4c19-ab80-384642433dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.399016 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85503aaf-d447-4327-8a0f-08232ce31e63 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.399398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85503aaf-d447-4327-8a0f-08232ce31e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.399733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85503aaf-d447-4327-8a0f-08232ce31e63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.400429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85503aaf-d447-4327-8a0f-08232ce31e63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.400539 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa3101-3f36-4c19-ab80-384642433dfb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.402479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85503aaf-d447-4327-8a0f-08232ce31e63 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.415753 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1020/2041] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:15:40.426107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa3101-3f36-4c19-ab80-384642433dfb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:40.432294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1022/2042] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:40 2022] POST /identity/v3/projects => generated 359 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:15:40.435952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa3101-3f36-4c19-ab80-384642433dfb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:40.436190 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85503aaf-d447-4327-8a0f-08232ce31e63 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:40.449721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:40.509115 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:40.514248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460e319-8893-482b-80e9-0efcbc28f885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.515082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460e319-8893-482b-80e9-0efcbc28f885 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.515358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460e319-8893-482b-80e9-0efcbc28f885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.515627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460e319-8893-482b-80e9-0efcbc28f885 admin admin] PATH_INFO: `/v3/projects/93630b505e70436dbbb5fb3337b974c5/users/98aa7cd13484414f8f32104bd91072e7/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.518791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460e319-8893-482b-80e9-0efcbc28f885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93630b505e70436dbbb5fb3337b974c5, user_id=98aa7cd13484414f8f32104bd91072e7, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.521599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460e319-8893-482b-80e9-0efcbc28f885 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.525208 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85503aaf-d447-4327-8a0f-08232ce31e63 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:40.530438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f637d-2566-4265-b586-aa157a3d8398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.531312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f637d-2566-4265-b586-aa157a3d8398 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.531756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f637d-2566-4265-b586-aa157a3d8398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.532026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f637d-2566-4265-b586-aa157a3d8398 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.532631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1021/2043] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:40 2022] PUT /identity/v3/projects/93630b505e70436dbbb5fb3337b974c5/users/98aa7cd13484414f8f32104bd91072e7/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:15:40.532781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f637d-2566-4265-b586-aa157a3d8398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.542733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f637d-2566-4265-b586-aa157a3d8398 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.545097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01391bee-e860-4eed-9fe6-d86da8fcc898 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.545380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01391bee-e860-4eed-9fe6-d86da8fcc898 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.545649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01391bee-e860-4eed-9fe6-d86da8fcc898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.568536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1023/2044] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:40 2022] POST /identity/v3/users => generated 280 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:15:40.572641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f637d-2566-4265-b586-aa157a3d8398 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:40.575575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-01391bee-e860-4eed-9fe6-d86da8fcc898 None None] MFA Rules not processed for user `98aa7cd13484414f8f32104bd91072e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:40.576800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-01391bee-e860-4eed-9fe6-d86da8fcc898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:40.586706 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f637d-2566-4265-b586-aa157a3d8398 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:40.647825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75399cd0-f467-4f1b-baf1-b8dedc978070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.648594 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75399cd0-f467-4f1b-baf1-b8dedc978070 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.648734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1022/2045] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:40.648845 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75399cd0-f467-4f1b-baf1-b8dedc978070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.648947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75399cd0-f467-4f1b-baf1-b8dedc978070 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.649548 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75399cd0-f467-4f1b-baf1-b8dedc978070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.651142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75399cd0-f467-4f1b-baf1-b8dedc978070 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.661506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01391bee-e860-4eed-9fe6-d86da8fcc898 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:40.663091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-01391bee-e860-4eed-9fe6-d86da8fcc898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:40.664038 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1024/2046] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:40.668094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75399cd0-f467-4f1b-baf1-b8dedc978070 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:40.678639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75399cd0-f467-4f1b-baf1-b8dedc978070 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:40.683673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c3e84-ab34-45f4-8d7e-274777224e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.683920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c3e84-ab34-45f4-8d7e-274777224e03 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.684188 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c3e84-ab34-45f4-8d7e-274777224e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.684455 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c3e84-ab34-45f4-8d7e-274777224e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.685130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c3e84-ab34-45f4-8d7e-274777224e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.686722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c3e84-ab34-45f4-8d7e-274777224e03 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.697856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1025/2047] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:15:40.702102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c3e84-ab34-45f4-8d7e-274777224e03 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:40.713134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c3e84-ab34-45f4-8d7e-274777224e03 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:40.717781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4ffaa-c15a-4b17-89eb-076f2c904227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.718445 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4ffaa-c15a-4b17-89eb-076f2c904227 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.718724 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4ffaa-c15a-4b17-89eb-076f2c904227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.718992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4ffaa-c15a-4b17-89eb-076f2c904227 admin admin] PATH_INFO: `/v3/projects/82e38490d3d84038974ad5810798d096/users/37021b8d898b4bc68cd7036f974c3eba/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.721794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01391bee-e860-4eed-9fe6-d86da8fcc898 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:40.725614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113d21b-a038-4378-a646-a6e7c74fce05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:40.726231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113d21b-a038-4378-a646-a6e7c74fce05 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.726504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113d21b-a038-4378-a646-a6e7c74fce05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.726769 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113d21b-a038-4378-a646-a6e7c74fce05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.728759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d113d21b-a038-4378-a646-a6e7c74fce05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:40.734234 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4ffaa-c15a-4b17-89eb-076f2c904227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e38490d3d84038974ad5810798d096, user_id=37021b8d898b4bc68cd7036f974c3eba, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.736926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4ffaa-c15a-4b17-89eb-076f2c904227 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.752401 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1026/2048] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:40 2022] PUT /identity/v3/projects/82e38490d3d84038974ad5810798d096/users/37021b8d898b4bc68cd7036f974c3eba/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:15:40.764592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b2bc-2ba8-4a81-aab1-6fccd3c7b01f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:40.765086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b2bc-2ba8-4a81-aab1-6fccd3c7b01f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:40.765492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b2bc-2ba8-4a81-aab1-6fccd3c7b01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:40.786506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:40.797470 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5f68b2bc-2ba8-4a81-aab1-6fccd3c7b01f None None] MFA Rules not processed for user `37021b8d898b4bc68cd7036f974c3eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:40.798342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113d21b-a038-4378-a646-a6e7c74fce05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:40.798730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5f68b2bc-2ba8-4a81-aab1-6fccd3c7b01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:40.800095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113d21b-a038-4378-a646-a6e7c74fce05 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:40.809315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1023/2049] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:15:40.873050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1027/2050] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:40 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:40.932222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113d21b-a038-4378-a646-a6e7c74fce05 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:40.933781 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d113d21b-a038-4378-a646-a6e7c74fce05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:40.946172 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113d21b-a038-4378-a646-a6e7c74fce05 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:41.003363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c28226-e7af-42b5-a33c-d0e7206c0609 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:41.004043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c28226-e7af-42b5-a33c-d0e7206c0609 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:41.004332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c28226-e7af-42b5-a33c-d0e7206c0609 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:41.004606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c28226-e7af-42b5-a33c-d0e7206c0609 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:41.015194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c28226-e7af-42b5-a33c-d0e7206c0609 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:41.016842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c28226-e7af-42b5-a33c-d0e7206c0609 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:41.027915 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1024/2051] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:40 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:41.046099 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68b2bc-2ba8-4a81-aab1-6fccd3c7b01f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:41.084068 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:41.125216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68b2bc-2ba8-4a81-aab1-6fccd3c7b01f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:41.130129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:41.130645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:41.130919 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:41.131186 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] PATH_INFO: `/v3/users/37021b8d898b4bc68cd7036f974c3eba` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:41.132295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37021b8d898b4bc68cd7036f974c3eba)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:41.134337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:41.190149 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37021b8d898b4bc68cd7036f974c3eba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:41.190838 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37021b8d898b4bc68cd7036f974c3eba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:41.198361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37021b8d898b4bc68cd7036f974c3eba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:41.210424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37021b8d898b4bc68cd7036f974c3eba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:41.211098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37021b8d898b4bc68cd7036f974c3eba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:41.226137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1028/2052] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:41 2022] DELETE /identity/v3/users/37021b8d898b4bc68cd7036f974c3eba => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:41.232099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe05f24-f28c-424b-b8e3-d4a022844e70 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:41.232384 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe05f24-f28c-424b-b8e3-d4a022844e70 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:41.232680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe05f24-f28c-424b-b8e3-d4a022844e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:41.251716 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bfe05f24-f28c-424b-b8e3-d4a022844e70 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:41.252999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bfe05f24-f28c-424b-b8e3-d4a022844e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:41.314646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1025/2053] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:41.324772 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:41.326506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:41.394429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a32857-b1bb-445d-890b-c07d983f8d46 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:41.400849 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee442e11-9187-4382-957d-dfae8bc94f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:41.401390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee442e11-9187-4382-957d-dfae8bc94f4c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:41.401665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee442e11-9187-4382-957d-dfae8bc94f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:41.401929 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee442e11-9187-4382-957d-dfae8bc94f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:41.413824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee442e11-9187-4382-957d-dfae8bc94f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:41.415275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee442e11-9187-4382-957d-dfae8bc94f4c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:41.425981 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1029/2054] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:41.813780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe05f24-f28c-424b-b8e3-d4a022844e70 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:41.815325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bfe05f24-f28c-424b-b8e3-d4a022844e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:41.827759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe05f24-f28c-424b-b8e3-d4a022844e70 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:41.832847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9271479-52ce-4a19-8ffd-32817915f8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:41.833505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9271479-52ce-4a19-8ffd-32817915f8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:41.833784 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9271479-52ce-4a19-8ffd-32817915f8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:41.834042 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9271479-52ce-4a19-8ffd-32817915f8c0 admin admin] PATH_INFO: `/v3/projects/82e38490d3d84038974ad5810798d096` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:41.835134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9271479-52ce-4a19-8ffd-32817915f8c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82e38490d3d84038974ad5810798d096)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:41.837134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9271479-52ce-4a19-8ffd-32817915f8c0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:41.886127 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a9271479-52ce-4a19-8ffd-32817915f8c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82e38490d3d84038974ad5810798d096', 'request_id': 'req-a9271479-52ce-4a19-8ffd-32817915f8c0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:41.892377 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a9271479-52ce-4a19-8ffd-32817915f8c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82e38490d3d84038974ad5810798d096', 'request_id': 'req-a9271479-52ce-4a19-8ffd-32817915f8c0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:41.898209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a9271479-52ce-4a19-8ffd-32817915f8c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82e38490d3d84038974ad5810798d096', 'request_id': 'req-a9271479-52ce-4a19-8ffd-32817915f8c0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:41.912490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a9271479-52ce-4a19-8ffd-32817915f8c0 admin admin] The token cache is being invalidate because project 82e38490d3d84038974ad5810798d096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:41.912809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a9271479-52ce-4a19-8ffd-32817915f8c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:41.913452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a9271479-52ce-4a19-8ffd-32817915f8c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:41.915335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1026/2055] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:41 2022] DELETE /identity/v3/projects/82e38490d3d84038974ad5810798d096 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:41.920480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1889747-7c63-4aae-8087-6c0b0bab1450 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:41.920829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1889747-7c63-4aae-8087-6c0b0bab1450 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:41.921109 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1889747-7c63-4aae-8087-6c0b0bab1450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:41.942746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b1889747-7c63-4aae-8087-6c0b0bab1450 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:41.944000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b1889747-7c63-4aae-8087-6c0b0bab1450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:42.000070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:42.015076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1030/2056] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:42.023257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbaa77-0b54-4af7-87ee-8affc3ff1586 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.023542 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbaa77-0b54-4af7-87ee-8affc3ff1586 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.023842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbaa77-0b54-4af7-87ee-8affc3ff1586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.045561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-80bbaa77-0b54-4af7-87ee-8affc3ff1586 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:42.046786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-80bbaa77-0b54-4af7-87ee-8affc3ff1586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:42.056143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1027/2057] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:42.060623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1889747-7c63-4aae-8087-6c0b0bab1450 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:42.070734 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1889747-7c63-4aae-8087-6c0b0bab1450 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:42.075567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1e98f-4bf9-4672-b8ee-a27744594468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:42.076161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1e98f-4bf9-4672-b8ee-a27744594468 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.076458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1e98f-4bf9-4672-b8ee-a27744594468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.076759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1e98f-4bf9-4672-b8ee-a27744594468 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.077386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1e98f-4bf9-4672-b8ee-a27744594468 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:15:42.077681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1e98f-4bf9-4672-b8ee-a27744594468 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:42.079279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1e98f-4bf9-4672-b8ee-a27744594468 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:42.091203 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1031/2058] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:15:42.097824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbaa77-0b54-4af7-87ee-8affc3ff1586 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:42.107595 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbaa77-0b54-4af7-87ee-8affc3ff1586 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:42.113190 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee00c22-69fb-417a-870e-daa5aeec2e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:42.113784 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee00c22-69fb-417a-870e-daa5aeec2e5f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.114052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee00c22-69fb-417a-870e-daa5aeec2e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.114319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee00c22-69fb-417a-870e-daa5aeec2e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.114964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee00c22-69fb-417a-870e-daa5aeec2e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:42.116947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee00c22-69fb-417a-870e-daa5aeec2e5f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:42.122351 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:42.152011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1028/2059] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:42 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:42.156279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1e98f-4bf9-4672-b8ee-a27744594468 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:42.168411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1e98f-4bf9-4672-b8ee-a27744594468 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:42.230457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d3439-6d83-4a5b-8684-c87fc9cdaa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:42.231068 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d3439-6d83-4a5b-8684-c87fc9cdaa83 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.231342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d3439-6d83-4a5b-8684-c87fc9cdaa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.231604 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d3439-6d83-4a5b-8684-c87fc9cdaa83 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.232229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d3439-6d83-4a5b-8684-c87fc9cdaa83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:42.234183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d3439-6d83-4a5b-8684-c87fc9cdaa83 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:42.258515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1032/2060] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:42 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:42.262693 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee00c22-69fb-417a-870e-daa5aeec2e5f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:42.273743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee00c22-69fb-417a-870e-daa5aeec2e5f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:42.278572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14852c73-332b-488d-8265-101ae62a16b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:42.279218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14852c73-332b-488d-8265-101ae62a16b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.279495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14852c73-332b-488d-8265-101ae62a16b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.279767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14852c73-332b-488d-8265-101ae62a16b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.280422 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14852c73-332b-488d-8265-101ae62a16b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:42.282065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14852c73-332b-488d-8265-101ae62a16b6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:42.292494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1029/2061] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:42.296602 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d3439-6d83-4a5b-8684-c87fc9cdaa83 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:42.307886 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d3439-6d83-4a5b-8684-c87fc9cdaa83 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:42.313910 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319153ba-e46c-409a-b17e-127b5f533dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:42.314542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319153ba-e46c-409a-b17e-127b5f533dee admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.314850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319153ba-e46c-409a-b17e-127b5f533dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.315120 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319153ba-e46c-409a-b17e-127b5f533dee admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.315794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319153ba-e46c-409a-b17e-127b5f533dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:42.317550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319153ba-e46c-409a-b17e-127b5f533dee admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:42.328672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1033/2062] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:42.333104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14852c73-332b-488d-8265-101ae62a16b6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:42.346014 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14852c73-332b-488d-8265-101ae62a16b6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:42.352037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def7b36-5348-475b-b03f-45c00d5eb057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:42.352711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def7b36-5348-475b-b03f-45c00d5eb057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.352995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def7b36-5348-475b-b03f-45c00d5eb057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.353262 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def7b36-5348-475b-b03f-45c00d5eb057 admin admin] PATH_INFO: `/v3/projects/ab7a2261d05041f393712b84aaa7bd52/users/1e404413672d478e9eeb2e2f8c237349/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.369389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def7b36-5348-475b-b03f-45c00d5eb057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7a2261d05041f393712b84aaa7bd52, user_id=1e404413672d478e9eeb2e2f8c237349, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:42.372070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def7b36-5348-475b-b03f-45c00d5eb057 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:42.388017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1030/2063] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:42 2022] PUT /identity/v3/projects/ab7a2261d05041f393712b84aaa7bd52/users/1e404413672d478e9eeb2e2f8c237349/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:42.402546 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfd98f-6717-4daa-ba64-ed5d996115bd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.402834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfd98f-6717-4daa-ba64-ed5d996115bd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.403109 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfd98f-6717-4daa-ba64-ed5d996115bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.433058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-98bfd98f-6717-4daa-ba64-ed5d996115bd None None] MFA Rules not processed for user `1e404413672d478e9eeb2e2f8c237349`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:42.434353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-98bfd98f-6717-4daa-ba64-ed5d996115bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:42.509040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1034/2064] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:42.514564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def7b36-5348-475b-b03f-45c00d5eb057 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:42.582494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def7b36-5348-475b-b03f-45c00d5eb057 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:42.587470 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ac567-469b-436c-8a47-646bb590c174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:42.588087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ac567-469b-436c-8a47-646bb590c174 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.588362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ac567-469b-436c-8a47-646bb590c174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.588672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ac567-469b-436c-8a47-646bb590c174 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.589275 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ac567-469b-436c-8a47-646bb590c174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:42.591176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ac567-469b-436c-8a47-646bb590c174 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:42.621821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1031/2065] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:42 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:42.627268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfd98f-6717-4daa-ba64-ed5d996115bd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:42.639302 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfd98f-6717-4daa-ba64-ed5d996115bd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:42.689730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:42.703147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5079ad-ee74-4149-b24e-6acd5ed9b0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:42.703774 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5079ad-ee74-4149-b24e-6acd5ed9b0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.704050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5079ad-ee74-4149-b24e-6acd5ed9b0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.704313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5079ad-ee74-4149-b24e-6acd5ed9b0f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.704979 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5079ad-ee74-4149-b24e-6acd5ed9b0f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:42.707031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5079ad-ee74-4149-b24e-6acd5ed9b0f4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:42.729415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1035/2066] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:42 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:42.733620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ac567-469b-436c-8a47-646bb590c174 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:42.744528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ac567-469b-436c-8a47-646bb590c174 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:42.750001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d606f5-7e75-4019-a7dd-cfeb61e12a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:42.750726 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d606f5-7e75-4019-a7dd-cfeb61e12a76 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.751010 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d606f5-7e75-4019-a7dd-cfeb61e12a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.751283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d606f5-7e75-4019-a7dd-cfeb61e12a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.751937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d606f5-7e75-4019-a7dd-cfeb61e12a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:42.753624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d606f5-7e75-4019-a7dd-cfeb61e12a76 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:42.765320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1032/2067] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:42.771129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5079ad-ee74-4149-b24e-6acd5ed9b0f4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:42.782497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5079ad-ee74-4149-b24e-6acd5ed9b0f4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:42.788541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33821423-6879-4fba-bc39-21fca22abc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:42.789195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33821423-6879-4fba-bc39-21fca22abc72 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.789475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33821423-6879-4fba-bc39-21fca22abc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.789742 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33821423-6879-4fba-bc39-21fca22abc72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.790390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33821423-6879-4fba-bc39-21fca22abc72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:42.792017 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33821423-6879-4fba-bc39-21fca22abc72 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:42.804136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1036/2068] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:42.807554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d606f5-7e75-4019-a7dd-cfeb61e12a76 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:42.818388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d606f5-7e75-4019-a7dd-cfeb61e12a76 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:42.824758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9294fcd-aa96-4e10-8e1b-600cc98004b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:42.825415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9294fcd-aa96-4e10-8e1b-600cc98004b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.825702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9294fcd-aa96-4e10-8e1b-600cc98004b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.825968 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9294fcd-aa96-4e10-8e1b-600cc98004b5 admin admin] PATH_INFO: `/v3/projects/95ed7e9b2585468db7d68ee3282804f9/users/5774afe92302409db75006393174ceeb/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.842579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9294fcd-aa96-4e10-8e1b-600cc98004b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ed7e9b2585468db7d68ee3282804f9, user_id=5774afe92302409db75006393174ceeb, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:42.845895 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9294fcd-aa96-4e10-8e1b-600cc98004b5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:42.860455 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1033/2069] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:42 2022] PUT /identity/v3/projects/95ed7e9b2585468db7d68ee3282804f9/users/5774afe92302409db75006393174ceeb/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:42.870730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576520c-effc-4ed3-a8bb-75ba58ffb9cc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:42.871185 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576520c-effc-4ed3-a8bb-75ba58ffb9cc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:42.871423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576520c-effc-4ed3-a8bb-75ba58ffb9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:42.905125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4576520c-effc-4ed3-a8bb-75ba58ffb9cc None None] MFA Rules not processed for user `5774afe92302409db75006393174ceeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:42.906510 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4576520c-effc-4ed3-a8bb-75ba58ffb9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:42.978051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1037/2070] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:42.982444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9294fcd-aa96-4e10-8e1b-600cc98004b5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:43.044709 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9294fcd-aa96-4e10-8e1b-600cc98004b5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:43.049821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c5e1e-9e7e-4be7-95b5-9cb7f436c8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:43.050432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c5e1e-9e7e-4be7-95b5-9cb7f436c8c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:43.050739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c5e1e-9e7e-4be7-95b5-9cb7f436c8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:43.051012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c5e1e-9e7e-4be7-95b5-9cb7f436c8c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:43.051645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c5e1e-9e7e-4be7-95b5-9cb7f436c8c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:43.053615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c5e1e-9e7e-4be7-95b5-9cb7f436c8c5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:43.059046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:43.083169 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1034/2071] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:15:42 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:15:43.087299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576520c-effc-4ed3-a8bb-75ba58ffb9cc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:43.098809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576520c-effc-4ed3-a8bb-75ba58ffb9cc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:43.167804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8ce8e-963d-46b1-b074-b2105f7ad3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:43.168197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8ce8e-963d-46b1-b074-b2105f7ad3cc admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:43.168498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8ce8e-963d-46b1-b074-b2105f7ad3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:43.168813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8ce8e-963d-46b1-b074-b2105f7ad3cc admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:43.169639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8ce8e-963d-46b1-b074-b2105f7ad3cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:43.171574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8ce8e-963d-46b1-b074-b2105f7ad3cc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:43.198731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1038/2072] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:15:43 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:15:43.202700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c5e1e-9e7e-4be7-95b5-9cb7f436c8c5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:43.214652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c5e1e-9e7e-4be7-95b5-9cb7f436c8c5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:43.220345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b01f57-d3c8-483c-8b45-b5932ad18bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:43.220972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b01f57-d3c8-483c-8b45-b5932ad18bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:43.221253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b01f57-d3c8-483c-8b45-b5932ad18bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:43.221581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b01f57-d3c8-483c-8b45-b5932ad18bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:43.222426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b01f57-d3c8-483c-8b45-b5932ad18bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:43.224275 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b01f57-d3c8-483c-8b45-b5932ad18bd3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:43.235680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1035/2073] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:43.240189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8ce8e-963d-46b1-b074-b2105f7ad3cc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:43.250818 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8ce8e-963d-46b1-b074-b2105f7ad3cc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:43.255724 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b49ca0-4722-4ef3-9571-1fbaeb62aaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:43.256381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b49ca0-4722-4ef3-9571-1fbaeb62aaab admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:43.256684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b49ca0-4722-4ef3-9571-1fbaeb62aaab admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:43.256956 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b49ca0-4722-4ef3-9571-1fbaeb62aaab admin admin] PATH_INFO: `/v3/projects/099d9268f9184ab28e23e9246f07c3be/users/f77ebb0d6e824aa7b66a80a8fb904958/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:43.273271 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b49ca0-4722-4ef3-9571-1fbaeb62aaab admin admin] RBAC: Authorizing `identity:create_grant(project_id=099d9268f9184ab28e23e9246f07c3be, user_id=f77ebb0d6e824aa7b66a80a8fb904958, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:43.276690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b49ca0-4722-4ef3-9571-1fbaeb62aaab admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:43.292681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1039/2074] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:43 2022] PUT /identity/v3/projects/099d9268f9184ab28e23e9246f07c3be/users/f77ebb0d6e824aa7b66a80a8fb904958/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:43.297017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b01f57-d3c8-483c-8b45-b5932ad18bd3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:43.308022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b01f57-d3c8-483c-8b45-b5932ad18bd3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:43.366956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7ce5d-2747-4b3a-a316-5456e1c846dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:43.367574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7ce5d-2747-4b3a-a316-5456e1c846dc admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:43.367852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7ce5d-2747-4b3a-a316-5456e1c846dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:43.368118 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7ce5d-2747-4b3a-a316-5456e1c846dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:43.368800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7ce5d-2747-4b3a-a316-5456e1c846dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:43.370443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7ce5d-2747-4b3a-a316-5456e1c846dc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:43.380893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1036/2075] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:15:43.385567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b49ca0-4722-4ef3-9571-1fbaeb62aaab admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:43.396458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b49ca0-4722-4ef3-9571-1fbaeb62aaab admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:43.401400 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7660994-1cde-4375-bc4f-915ac33e0198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:43.402049 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7660994-1cde-4375-bc4f-915ac33e0198 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:43.402324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7660994-1cde-4375-bc4f-915ac33e0198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:43.402835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7660994-1cde-4375-bc4f-915ac33e0198 admin admin] PATH_INFO: `/v3/projects/099d9268f9184ab28e23e9246f07c3be/users/f77ebb0d6e824aa7b66a80a8fb904958/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:43.404618 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7660994-1cde-4375-bc4f-915ac33e0198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=099d9268f9184ab28e23e9246f07c3be, user_id=f77ebb0d6e824aa7b66a80a8fb904958, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:43.407296 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7660994-1cde-4375-bc4f-915ac33e0198 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:43.417921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1040/2076] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:15:43 2022] PUT /identity/v3/projects/099d9268f9184ab28e23e9246f07c3be/users/f77ebb0d6e824aa7b66a80a8fb904958/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:43.432185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0376ca3-c683-407e-bbd2-34ddb818dec3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:43.432540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0376ca3-c683-407e-bbd2-34ddb818dec3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:43.432961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0376ca3-c683-407e-bbd2-34ddb818dec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:43.468903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b0376ca3-c683-407e-bbd2-34ddb818dec3 None None] MFA Rules not processed for user `f77ebb0d6e824aa7b66a80a8fb904958`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:43.470121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b0376ca3-c683-407e-bbd2-34ddb818dec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:43.545789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1037/2077] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:43.558388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7660994-1cde-4375-bc4f-915ac33e0198 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:43.560856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c7660994-1cde-4375-bc4f-915ac33e0198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:43.573091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:43.629092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7660994-1cde-4375-bc4f-915ac33e0198 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:43.633095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f8709-4c8b-4bb3-8fa9-cdb39d58ef3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:43.633708 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f8709-4c8b-4bb3-8fa9-cdb39d58ef3e service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:43.634009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f8709-4c8b-4bb3-8fa9-cdb39d58ef3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:43.634253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f8709-4c8b-4bb3-8fa9-cdb39d58ef3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:43.706544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f8709-4c8b-4bb3-8fa9-cdb39d58ef3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:43.709014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f8709-4c8b-4bb3-8fa9-cdb39d58ef3e service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:43.724731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1041/2078] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:46.636496 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0376ca3-c683-407e-bbd2-34ddb818dec3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:46.649481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0376ca3-c683-407e-bbd2-34ddb818dec3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:46.653261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a667cca-c61b-4193-aa02-7e8a5be53323 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:46.653937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a667cca-c61b-4193-aa02-7e8a5be53323 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:46.654219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a667cca-c61b-4193-aa02-7e8a5be53323 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:46.654494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a667cca-c61b-4193-aa02-7e8a5be53323 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:46.656296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4a667cca-c61b-4193-aa02-7e8a5be53323 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:46.719009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:46.721732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a667cca-c61b-4193-aa02-7e8a5be53323 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:46.723809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a667cca-c61b-4193-aa02-7e8a5be53323 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:46.734157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1038/2079] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:47.407791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f8709-4c8b-4bb3-8fa9-cdb39d58ef3e service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:47.409217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5f3f8709-4c8b-4bb3-8fa9-cdb39d58ef3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:47.420681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f8709-4c8b-4bb3-8fa9-cdb39d58ef3e service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:47.425717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43ca3a-86a3-4b19-b80b-eccad1003ed9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:47.426315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43ca3a-86a3-4b19-b80b-eccad1003ed9 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:47.426590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43ca3a-86a3-4b19-b80b-eccad1003ed9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:47.426861 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43ca3a-86a3-4b19-b80b-eccad1003ed9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:47.493925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43ca3a-86a3-4b19-b80b-eccad1003ed9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:47.495371 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43ca3a-86a3-4b19-b80b-eccad1003ed9 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:47.507796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1042/2080] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:50.241444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a667cca-c61b-4193-aa02-7e8a5be53323 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:50.250488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a667cca-c61b-4193-aa02-7e8a5be53323 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:50.255470 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea143f6-f793-41f0-9665-b114c86abf38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:50.256079 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea143f6-f793-41f0-9665-b114c86abf38 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:50.256353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea143f6-f793-41f0-9665-b114c86abf38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:50.256647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea143f6-f793-41f0-9665-b114c86abf38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:50.258893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1ea143f6-f793-41f0-9665-b114c86abf38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:50.329054 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea143f6-f793-41f0-9665-b114c86abf38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:50.330507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea143f6-f793-41f0-9665-b114c86abf38 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:50.340566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1039/2081] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:52.025140 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43ca3a-86a3-4b19-b80b-eccad1003ed9 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:52.034586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43ca3a-86a3-4b19-b80b-eccad1003ed9 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:52.038004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17667794-ea84-4c32-8c9d-df9a1d0d5281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:52.038610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17667794-ea84-4c32-8c9d-df9a1d0d5281 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:52.038887 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17667794-ea84-4c32-8c9d-df9a1d0d5281 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:52.039155 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17667794-ea84-4c32-8c9d-df9a1d0d5281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:52.048642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17667794-ea84-4c32-8c9d-df9a1d0d5281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:52.050519 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17667794-ea84-4c32-8c9d-df9a1d0d5281 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:52.060008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1043/2082] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:57.517331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea143f6-f793-41f0-9665-b114c86abf38 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:57.518937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1ea143f6-f793-41f0-9665-b114c86abf38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:57.589157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea143f6-f793-41f0-9665-b114c86abf38 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:57.593807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:57.594443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:57.594721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:57.594993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] PATH_INFO: `/v3/users/a2ce2bb2f2ca449884bab2cf432d20de` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:57.596119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2ce2bb2f2ca449884bab2cf432d20de)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:57.598088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:57.649952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2ce2bb2f2ca449884bab2cf432d20de'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:57.650932 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2ce2bb2f2ca449884bab2cf432d20de'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:57.658184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2ce2bb2f2ca449884bab2cf432d20de'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:57.672059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2ce2bb2f2ca449884bab2cf432d20de'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:57.672957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2ce2bb2f2ca449884bab2cf432d20de'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:57.694184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1040/2083] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:57 2022] DELETE /identity/v3/users/a2ce2bb2f2ca449884bab2cf432d20de => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:57.699635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8cf83-7cf5-4364-b31e-22705af452ce None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:57.700136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8cf83-7cf5-4364-b31e-22705af452ce None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:57.700136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8cf83-7cf5-4364-b31e-22705af452ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:57.720691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0bb8cf83-7cf5-4364-b31e-22705af452ce None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:57.721942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0bb8cf83-7cf5-4364-b31e-22705af452ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:57.769042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:57.788460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1044/2084] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:57.798770 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:57.800844 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:57.814342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:57.865092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e423a6c-aab1-4255-8bab-6b8d5f77efe8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:57.869261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f34a07-3fe6-4831-bf15-f68789af725f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:57.869933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f34a07-3fe6-4831-bf15-f68789af725f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:57.870209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f34a07-3fe6-4831-bf15-f68789af725f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:57.870472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f34a07-3fe6-4831-bf15-f68789af725f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:57.880739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f34a07-3fe6-4831-bf15-f68789af725f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:57.882204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f34a07-3fe6-4831-bf15-f68789af725f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:57.892038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1041/2085] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:58.176468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8cf83-7cf5-4364-b31e-22705af452ce None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:58.178080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0bb8cf83-7cf5-4364-b31e-22705af452ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:58.191312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8cf83-7cf5-4364-b31e-22705af452ce None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:58.196891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:58.197511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:58.197836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:58.198106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] PATH_INFO: `/v3/projects/76a0436f87634bc9bbe55b249305503e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:58.199508 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] RBAC: Authorizing `identity:delete_project(project_id=76a0436f87634bc9bbe55b249305503e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:58.201502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:58.260934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76a0436f87634bc9bbe55b249305503e', 'request_id': 'req-d5c0b184-69e5-4057-baca-49f047453acb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:58.270364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76a0436f87634bc9bbe55b249305503e', 'request_id': 'req-d5c0b184-69e5-4057-baca-49f047453acb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:58.278628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76a0436f87634bc9bbe55b249305503e', 'request_id': 'req-d5c0b184-69e5-4057-baca-49f047453acb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:58.290165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] The token cache is being invalidate because project 76a0436f87634bc9bbe55b249305503e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:58.290492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:58.291205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:58.293159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1045/2086] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:58 2022] DELETE /identity/v3/projects/76a0436f87634bc9bbe55b249305503e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:58.297716 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f34a07-3fe6-4831-bf15-f68789af725f service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:58.308623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f34a07-3fe6-4831-bf15-f68789af725f service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:58.371737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:58.372484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:58.372866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:58.373140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] PATH_INFO: `/v3/users/c57bdc898edc4fcf8cd428044de97973` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:58.374425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c57bdc898edc4fcf8cd428044de97973)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:58.376774 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:58.451723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c57bdc898edc4fcf8cd428044de97973'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:58.451723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c57bdc898edc4fcf8cd428044de97973'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:58.460406 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c57bdc898edc4fcf8cd428044de97973'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:58.476684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c57bdc898edc4fcf8cd428044de97973'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:58.477399 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c57bdc898edc4fcf8cd428044de97973'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:58.505760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1042/2087] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:58 2022] DELETE /identity/v3/users/c57bdc898edc4fcf8cd428044de97973 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:58.845513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:58.847005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:58.862255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0b184-69e5-4057-baca-49f047453acb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:58.930759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:58.931217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:58.931498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:58.931770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] PATH_INFO: `/v3/projects/254fef49f402453a9e51dfbe60eb740f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:58.933081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] RBAC: Authorizing `identity:delete_project(project_id=254fef49f402453a9e51dfbe60eb740f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:58.935191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:58.992948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '254fef49f402453a9e51dfbe60eb740f', 'request_id': 'req-97fc51b4-39f2-422d-af3b-fcc5650bcdef'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.009635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '254fef49f402453a9e51dfbe60eb740f', 'request_id': 'req-97fc51b4-39f2-422d-af3b-fcc5650bcdef'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.018963 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '254fef49f402453a9e51dfbe60eb740f', 'request_id': 'req-97fc51b4-39f2-422d-af3b-fcc5650bcdef'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.029520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] The token cache is being invalidate because project 254fef49f402453a9e51dfbe60eb740f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:59.030456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.031197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.033127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1046/2088] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:58 2022] DELETE /identity/v3/projects/254fef49f402453a9e51dfbe60eb740f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:59.037141 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:59.038659 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:59.053278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d00de-2dcf-405c-852a-132991e9f3e6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:59.085007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:59.123091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585208a-0fa6-4c07-929e-7a1475b2c0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:59.123749 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585208a-0fa6-4c07-929e-7a1475b2c0c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:59.124029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585208a-0fa6-4c07-929e-7a1475b2c0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:59.124301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585208a-0fa6-4c07-929e-7a1475b2c0c8 admin admin] PATH_INFO: `/v3/users/98aa7cd13484414f8f32104bd91072e7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:59.125522 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585208a-0fa6-4c07-929e-7a1475b2c0c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98aa7cd13484414f8f32104bd91072e7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:59.127464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585208a-0fa6-4c07-929e-7a1475b2c0c8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:59.181927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8585208a-0fa6-4c07-929e-7a1475b2c0c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98aa7cd13484414f8f32104bd91072e7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.182635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8585208a-0fa6-4c07-929e-7a1475b2c0c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98aa7cd13484414f8f32104bd91072e7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.190484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8585208a-0fa6-4c07-929e-7a1475b2c0c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98aa7cd13484414f8f32104bd91072e7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.205024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8585208a-0fa6-4c07-929e-7a1475b2c0c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98aa7cd13484414f8f32104bd91072e7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.205767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8585208a-0fa6-4c07-929e-7a1475b2c0c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98aa7cd13484414f8f32104bd91072e7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.221088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1043/2089] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:59 2022] DELETE /identity/v3/users/98aa7cd13484414f8f32104bd91072e7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:59.303125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:59.305209 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:59.315550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:59.324330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc51b4-39f2-422d-af3b-fcc5650bcdef admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:59.390829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:59.391626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:59.391960 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:59.392308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] PATH_INFO: `/v3/users/1e404413672d478e9eeb2e2f8c237349` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:59.394268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e404413672d478e9eeb2e2f8c237349)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:59.397096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:59.453808 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e404413672d478e9eeb2e2f8c237349'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.454476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e404413672d478e9eeb2e2f8c237349'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.464854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e404413672d478e9eeb2e2f8c237349'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.485297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e404413672d478e9eeb2e2f8c237349'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.486463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e404413672d478e9eeb2e2f8c237349'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.510181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1047/2090] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:15:59 2022] DELETE /identity/v3/users/1e404413672d478e9eeb2e2f8c237349 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:59.512583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6467041-b459-4ef4-b6af-969494d79628 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:59.512902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6467041-b459-4ef4-b6af-969494d79628 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:59.513175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6467041-b459-4ef4-b6af-969494d79628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:59.534854 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c6467041-b459-4ef4-b6af-969494d79628 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:59.536022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c6467041-b459-4ef4-b6af-969494d79628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:59.545298 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:59.547578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:59.641690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1044/2091] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:15:59.652880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6467041-b459-4ef4-b6af-969494d79628 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:15:59.656315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c6467041-b459-4ef4-b6af-969494d79628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:59.670929 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e0a35-5a27-4e22-9565-f765a85a39bf admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:59.689590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374ff4f-8083-4726-aa12-4a5386f1382e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:59.690257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374ff4f-8083-4726-aa12-4a5386f1382e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:59.690536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374ff4f-8083-4726-aa12-4a5386f1382e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:59.690807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374ff4f-8083-4726-aa12-4a5386f1382e admin admin] PATH_INFO: `/v3/projects/93630b505e70436dbbb5fb3337b974c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:59.693820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374ff4f-8083-4726-aa12-4a5386f1382e admin admin] RBAC: Authorizing `identity:delete_project(project_id=93630b505e70436dbbb5fb3337b974c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:59.695771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374ff4f-8083-4726-aa12-4a5386f1382e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:59.729888 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:59.739477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6467041-b459-4ef4-b6af-969494d79628 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:15:59.756126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67b6e1-d638-4508-82b3-e11db3779475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:15:59.756955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67b6e1-d638-4508-82b3-e11db3779475 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:59.757334 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67b6e1-d638-4508-82b3-e11db3779475 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:59.757709 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67b6e1-d638-4508-82b3-e11db3779475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:59.764851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:15:59.769204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67b6e1-d638-4508-82b3-e11db3779475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:15:59.771283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67b6e1-d638-4508-82b3-e11db3779475 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:15:59.774192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7374ff4f-8083-4726-aa12-4a5386f1382e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93630b505e70436dbbb5fb3337b974c5', 'request_id': 'req-7374ff4f-8083-4726-aa12-4a5386f1382e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.783098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1045/2092] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:15:59.783911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7374ff4f-8083-4726-aa12-4a5386f1382e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93630b505e70436dbbb5fb3337b974c5', 'request_id': 'req-7374ff4f-8083-4726-aa12-4a5386f1382e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.791180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7374ff4f-8083-4726-aa12-4a5386f1382e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93630b505e70436dbbb5fb3337b974c5', 'request_id': 'req-7374ff4f-8083-4726-aa12-4a5386f1382e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.803360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7374ff4f-8083-4726-aa12-4a5386f1382e admin admin] The token cache is being invalidate because project 93630b505e70436dbbb5fb3337b974c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:15:59.803716 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7374ff4f-8083-4726-aa12-4a5386f1382e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.806562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7374ff4f-8083-4726-aa12-4a5386f1382e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:15:59.810836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1048/2093] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:15:59 2022] DELETE /identity/v3/projects/93630b505e70436dbbb5fb3337b974c5 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:15:59.815802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66336013-6084-4bce-8303-07118dd0a5ec None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:59.816088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66336013-6084-4bce-8303-07118dd0a5ec None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:59.816645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66336013-6084-4bce-8303-07118dd0a5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:59.841312 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-66336013-6084-4bce-8303-07118dd0a5ec None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:59.842755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-66336013-6084-4bce-8303-07118dd0a5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:59.931448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1046/2094] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:59.941259 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cdd51-e67e-4552-baad-4f337d1824f3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:15:59.941928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cdd51-e67e-4552-baad-4f337d1824f3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:15:59.942039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cdd51-e67e-4552-baad-4f337d1824f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:15:59.966945 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7f2cdd51-e67e-4552-baad-4f337d1824f3 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:15:59.971265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7f2cdd51-e67e-4552-baad-4f337d1824f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:15:59.978917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1049/2095] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:15:59.985763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66336013-6084-4bce-8303-07118dd0a5ec None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.013038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66336013-6084-4bce-8303-07118dd0a5ec None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.017730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa0132-e452-45e6-ad98-b2a3ebed6657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.018371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa0132-e452-45e6-ad98-b2a3ebed6657 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.018655 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa0132-e452-45e6-ad98-b2a3ebed6657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.018906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa0132-e452-45e6-ad98-b2a3ebed6657 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.019551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa0132-e452-45e6-ad98-b2a3ebed6657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:16:00.019844 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa0132-e452-45e6-ad98-b2a3ebed6657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:00.021523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa0132-e452-45e6-ad98-b2a3ebed6657 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:00.033535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1047/2096] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:16:00.038052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cdd51-e67e-4552-baad-4f337d1824f3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.048126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cdd51-e67e-4552-baad-4f337d1824f3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.053134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72736bd-e3af-4d2a-b231-2420011d7165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.053745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72736bd-e3af-4d2a-b231-2420011d7165 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.054019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72736bd-e3af-4d2a-b231-2420011d7165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.054284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72736bd-e3af-4d2a-b231-2420011d7165 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.054926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72736bd-e3af-4d2a-b231-2420011d7165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:00.056959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72736bd-e3af-4d2a-b231-2420011d7165 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:00.088079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1050/2097] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:00 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:00.093353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa0132-e452-45e6-ad98-b2a3ebed6657 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.105225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa0132-e452-45e6-ad98-b2a3ebed6657 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.109628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72736bd-e3af-4d2a-b231-2420011d7165 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.111801 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c72736bd-e3af-4d2a-b231-2420011d7165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:00.166037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7e251-d5ba-4076-b587-954f964ec221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.166692 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7e251-d5ba-4076-b587-954f964ec221 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.166964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7e251-d5ba-4076-b587-954f964ec221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.167228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7e251-d5ba-4076-b587-954f964ec221 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.168094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7e251-d5ba-4076-b587-954f964ec221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:00.169824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7e251-d5ba-4076-b587-954f964ec221 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:00.185884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72736bd-e3af-4d2a-b231-2420011d7165 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.195132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1048/2098] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:00 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:00.197192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.197927 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.198168 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.198438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] PATH_INFO: `/v3/projects/ab7a2261d05041f393712b84aaa7bd52` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.199208 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7e251-d5ba-4076-b587-954f964ec221 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.199576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7a2261d05041f393712b84aaa7bd52)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:00.201545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:00.208765 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7e251-d5ba-4076-b587-954f964ec221 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.214819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671312f6-c49c-4f27-acb8-7a419af3a4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.215437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671312f6-c49c-4f27-acb8-7a419af3a4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.215760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671312f6-c49c-4f27-acb8-7a419af3a4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.216035 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671312f6-c49c-4f27-acb8-7a419af3a4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.216750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671312f6-c49c-4f27-acb8-7a419af3a4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:00.218368 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671312f6-c49c-4f27-acb8-7a419af3a4f6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:00.230522 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1049/2099] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:00.234590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671312f6-c49c-4f27-acb8-7a419af3a4f6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.244842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671312f6-c49c-4f27-acb8-7a419af3a4f6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.249701 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7a2261d05041f393712b84aaa7bd52', 'request_id': 'req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.258619 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7a2261d05041f393712b84aaa7bd52', 'request_id': 'req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.265164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7a2261d05041f393712b84aaa7bd52', 'request_id': 'req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.275774 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] The token cache is being invalidate because project ab7a2261d05041f393712b84aaa7bd52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:00.276102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.276785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.278568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1051/2100] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:00 2022] DELETE /identity/v3/projects/ab7a2261d05041f393712b84aaa7bd52 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:16:00.282531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.284449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:00.303738 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:00.309530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809240d5-d2db-4e2e-a07c-c086fbdbe418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.310142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809240d5-d2db-4e2e-a07c-c086fbdbe418 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.310420 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809240d5-d2db-4e2e-a07c-c086fbdbe418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.310713 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809240d5-d2db-4e2e-a07c-c086fbdbe418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.311329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809240d5-d2db-4e2e-a07c-c086fbdbe418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:00.313001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809240d5-d2db-4e2e-a07c-c086fbdbe418 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:00.314444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:00.325511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1050/2101] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:16:00.329723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809240d5-d2db-4e2e-a07c-c086fbdbe418 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.331422 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-809240d5-d2db-4e2e-a07c-c086fbdbe418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:00.343619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809240d5-d2db-4e2e-a07c-c086fbdbe418 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.351221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47ae45-2aee-476a-b3e6-117d43c5e4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.351881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47ae45-2aee-476a-b3e6-117d43c5e4eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.352156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47ae45-2aee-476a-b3e6-117d43c5e4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.352426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47ae45-2aee-476a-b3e6-117d43c5e4eb admin admin] PATH_INFO: `/v3/projects/375ebb311dfc4034b2565cbc16c18c2b/users/5982975fb70d47eca8ae9f148f81a2a5/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.358439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaf6a2-d66b-4b57-9a5e-d04345d187ee admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.363187 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.363847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.364102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.364390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] PATH_INFO: `/v3/users/5774afe92302409db75006393174ceeb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.365565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5774afe92302409db75006393174ceeb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:00.367532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:00.368540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47ae45-2aee-476a-b3e6-117d43c5e4eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=375ebb311dfc4034b2565cbc16c18c2b, user_id=5982975fb70d47eca8ae9f148f81a2a5, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:00.371495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47ae45-2aee-476a-b3e6-117d43c5e4eb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:00.385697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1051/2102] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:00 2022] PUT /identity/v3/projects/375ebb311dfc4034b2565cbc16c18c2b/users/5982975fb70d47eca8ae9f148f81a2a5/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:16:00.396822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b831144-68fc-4743-8de2-e7dd387b977e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.397174 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b831144-68fc-4743-8de2-e7dd387b977e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.397469 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b831144-68fc-4743-8de2-e7dd387b977e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.420155 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5774afe92302409db75006393174ceeb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.420973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5774afe92302409db75006393174ceeb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.427436 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3b831144-68fc-4743-8de2-e7dd387b977e None None] MFA Rules not processed for user `5982975fb70d47eca8ae9f148f81a2a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:00.428523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5774afe92302409db75006393174ceeb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.428681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3b831144-68fc-4743-8de2-e7dd387b977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:00.440907 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5774afe92302409db75006393174ceeb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.441588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5774afe92302409db75006393174ceeb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.455448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1052/2103] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:00 2022] DELETE /identity/v3/users/5774afe92302409db75006393174ceeb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:16:00.499327 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1052/2104] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:00.500730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.502387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:00.573968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915e06f-e929-4caa-878a-9ff59b570bf1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.579095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc09a3-7c7b-46f7-a2c7-37c0c7970390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.579721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc09a3-7c7b-46f7-a2c7-37c0c7970390 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.579996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc09a3-7c7b-46f7-a2c7-37c0c7970390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.580262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc09a3-7c7b-46f7-a2c7-37c0c7970390 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.580927 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc09a3-7c7b-46f7-a2c7-37c0c7970390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:00.585626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc09a3-7c7b-46f7-a2c7-37c0c7970390 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:00.622647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1053/2105] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:00 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:00.627037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b831144-68fc-4743-8de2-e7dd387b977e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.693024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b831144-68fc-4743-8de2-e7dd387b977e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.739004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc09a3-7c7b-46f7-a2c7-37c0c7970390 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.742669 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fafc09a3-7c7b-46f7-a2c7-37c0c7970390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:00.775169 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335f464-2fa4-4c3e-98e5-79a6bee18d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.776012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335f464-2fa4-4c3e-98e5-79a6bee18d76 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.776291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335f464-2fa4-4c3e-98e5-79a6bee18d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.776630 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335f464-2fa4-4c3e-98e5-79a6bee18d76 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.777284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335f464-2fa4-4c3e-98e5-79a6bee18d76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:00.779142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335f464-2fa4-4c3e-98e5-79a6bee18d76 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:00.792509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:00.799356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:00.806464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1053/2106] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:00 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:00.812874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335f464-2fa4-4c3e-98e5-79a6bee18d76 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.823879 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335f464-2fa4-4c3e-98e5-79a6bee18d76 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.832513 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214217d1-a50f-47af-a20b-a9629bb1a9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.833158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214217d1-a50f-47af-a20b-a9629bb1a9ac admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.833435 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214217d1-a50f-47af-a20b-a9629bb1a9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.833705 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214217d1-a50f-47af-a20b-a9629bb1a9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.834358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214217d1-a50f-47af-a20b-a9629bb1a9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:00.835986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214217d1-a50f-47af-a20b-a9629bb1a9ac admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:00.846423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1054/2107] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:00.850259 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc09a3-7c7b-46f7-a2c7-37c0c7970390 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.850656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214217d1-a50f-47af-a20b-a9629bb1a9ac admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.855745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.856395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.856705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.856974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] PATH_INFO: `/v3/projects/95ed7e9b2585468db7d68ee3282804f9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.858122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95ed7e9b2585468db7d68ee3282804f9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:00.860178 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:00.864175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214217d1-a50f-47af-a20b-a9629bb1a9ac admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.869090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54b414-b770-438a-b29a-64ff2a560296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.869834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54b414-b770-438a-b29a-64ff2a560296 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.870118 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54b414-b770-438a-b29a-64ff2a560296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.870486 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54b414-b770-438a-b29a-64ff2a560296 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.871119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54b414-b770-438a-b29a-64ff2a560296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:00.872894 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54b414-b770-438a-b29a-64ff2a560296 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:00.892888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1055/2108] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:16:00.894438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54b414-b770-438a-b29a-64ff2a560296 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.906934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54b414-b770-438a-b29a-64ff2a560296 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:00.917899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95ed7e9b2585468db7d68ee3282804f9', 'request_id': 'req-76785c98-ac24-48dc-9bf7-402116231727'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.928703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95ed7e9b2585468db7d68ee3282804f9', 'request_id': 'req-76785c98-ac24-48dc-9bf7-402116231727'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.937402 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95ed7e9b2585468db7d68ee3282804f9', 'request_id': 'req-76785c98-ac24-48dc-9bf7-402116231727'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.949315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] The token cache is being invalidate because project 95ed7e9b2585468db7d68ee3282804f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:00.949907 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.950326 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:00.951851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1054/2109] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:00 2022] DELETE /identity/v3/projects/95ed7e9b2585468db7d68ee3282804f9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:00.955576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:00.956978 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:00.981445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52b099-1201-45ce-94ac-c352c2c49c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:00.982277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b099-1201-45ce-94ac-c352c2c49c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:00.982654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b099-1201-45ce-94ac-c352c2c49c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:00.982996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b099-1201-45ce-94ac-c352c2c49c0d admin admin] PATH_INFO: `/v3/projects/65d63841670549a1b5cbcce285c7c172/users/27551682c7904696a057504c52272a6c/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:00.999251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52b099-1201-45ce-94ac-c352c2c49c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d63841670549a1b5cbcce285c7c172, user_id=27551682c7904696a057504c52272a6c, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:01.001957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52b099-1201-45ce-94ac-c352c2c49c0d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:01.017210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1056/2110] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:00 2022] PUT /identity/v3/projects/65d63841670549a1b5cbcce285c7c172/users/27551682c7904696a057504c52272a6c/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:16:01.028085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692f372-5b07-44a5-9db9-ceda55f8b5ba None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.028410 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692f372-5b07-44a5-9db9-ceda55f8b5ba None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.028647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692f372-5b07-44a5-9db9-ceda55f8b5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.032755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76785c98-ac24-48dc-9bf7-402116231727 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:01.063318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6692f372-5b07-44a5-9db9-ceda55f8b5ba None None] MFA Rules not processed for user `27551682c7904696a057504c52272a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:01.064491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6692f372-5b07-44a5-9db9-ceda55f8b5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:01.092577 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:01.101444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5629ced-734c-45d3-9f80-a6409626ee53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:01.102067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5629ced-734c-45d3-9f80-a6409626ee53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.102350 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5629ced-734c-45d3-9f80-a6409626ee53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.102901 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5629ced-734c-45d3-9f80-a6409626ee53 admin admin] PATH_INFO: `/v3/users/f77ebb0d6e824aa7b66a80a8fb904958` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.104171 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5629ced-734c-45d3-9f80-a6409626ee53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f77ebb0d6e824aa7b66a80a8fb904958)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:01.106502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5629ced-734c-45d3-9f80-a6409626ee53 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:01.131597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1057/2111] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:01.136268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692f372-5b07-44a5-9db9-ceda55f8b5ba None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:01.137800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6692f372-5b07-44a5-9db9-ceda55f8b5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:01.151756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692f372-5b07-44a5-9db9-ceda55f8b5ba None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:01.154125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e5629ced-734c-45d3-9f80-a6409626ee53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f77ebb0d6e824aa7b66a80a8fb904958'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:01.154773 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e5629ced-734c-45d3-9f80-a6409626ee53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f77ebb0d6e824aa7b66a80a8fb904958'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:01.156896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc2b5c-9b61-43e4-9a74-b428e5a59565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:01.157505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc2b5c-9b61-43e4-9a74-b428e5a59565 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.157806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc2b5c-9b61-43e4-9a74-b428e5a59565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.158247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc2b5c-9b61-43e4-9a74-b428e5a59565 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.158888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc2b5c-9b61-43e4-9a74-b428e5a59565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:01.160858 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc2b5c-9b61-43e4-9a74-b428e5a59565 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:01.161879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e5629ced-734c-45d3-9f80-a6409626ee53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f77ebb0d6e824aa7b66a80a8fb904958'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:01.174748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e5629ced-734c-45d3-9f80-a6409626ee53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f77ebb0d6e824aa7b66a80a8fb904958'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:01.175429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e5629ced-734c-45d3-9f80-a6409626ee53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f77ebb0d6e824aa7b66a80a8fb904958'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:01.193821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1058/2112] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:01 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:16:01.194018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1055/2113] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:00 2022] DELETE /identity/v3/users/f77ebb0d6e824aa7b66a80a8fb904958 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:16:01.202323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc2b5c-9b61-43e4-9a74-b428e5a59565 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:01.204358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-36bc2b5c-9b61-43e4-9a74-b428e5a59565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:01.238356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:01.275242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc2b5c-9b61-43e4-9a74-b428e5a59565 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:01.334945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cba301-ed85-4eb9-aab4-dadefd43995c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:01.335465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cba301-ed85-4eb9-aab4-dadefd43995c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.335737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cba301-ed85-4eb9-aab4-dadefd43995c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.336005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cba301-ed85-4eb9-aab4-dadefd43995c admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.336630 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cba301-ed85-4eb9-aab4-dadefd43995c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:01.339890 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cba301-ed85-4eb9-aab4-dadefd43995c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:01.367456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1059/2114] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:01 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:01.371398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5629ced-734c-45d3-9f80-a6409626ee53 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:01.388092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5629ced-734c-45d3-9f80-a6409626ee53 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:01.395764 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1293ee6-884c-442d-8feb-871ab43729ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:01.396416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1293ee6-884c-442d-8feb-871ab43729ed admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.396740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1293ee6-884c-442d-8feb-871ab43729ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.397094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1293ee6-884c-442d-8feb-871ab43729ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.397890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1293ee6-884c-442d-8feb-871ab43729ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:01.399559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1293ee6-884c-442d-8feb-871ab43729ed admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:01.411813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1056/2115] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:16:01.416264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cba301-ed85-4eb9-aab4-dadefd43995c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:01.426987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cba301-ed85-4eb9-aab4-dadefd43995c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:01.431883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1b8ce-35dc-4116-9930-d2ff4565f81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:01.432591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1b8ce-35dc-4116-9930-d2ff4565f81c admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.432894 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1b8ce-35dc-4116-9930-d2ff4565f81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.433170 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1b8ce-35dc-4116-9930-d2ff4565f81c admin admin] PATH_INFO: `/v3/projects/b0b7b50254344bd3ab120e546cc15b39/users/d6f37c6e1a104e4fa3409f674a4f52c3/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.448544 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1b8ce-35dc-4116-9930-d2ff4565f81c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b7b50254344bd3ab120e546cc15b39, user_id=d6f37c6e1a104e4fa3409f674a4f52c3, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:01.451253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1b8ce-35dc-4116-9930-d2ff4565f81c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:01.469182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1060/2116] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:01 2022] PUT /identity/v3/projects/b0b7b50254344bd3ab120e546cc15b39/users/d6f37c6e1a104e4fa3409f674a4f52c3/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:01.473323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1293ee6-884c-442d-8feb-871ab43729ed admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:01.477228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1b8ce-35dc-4116-9930-d2ff4565f81c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:01.478925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-27f1b8ce-35dc-4116-9930-d2ff4565f81c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:01.486233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1293ee6-884c-442d-8feb-871ab43729ed admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:01.551198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2c104-5d0d-4b0c-81d1-0f7de9c1878c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:01.551834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2c104-5d0d-4b0c-81d1-0f7de9c1878c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.552092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2c104-5d0d-4b0c-81d1-0f7de9c1878c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.552358 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2c104-5d0d-4b0c-81d1-0f7de9c1878c admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.553054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2c104-5d0d-4b0c-81d1-0f7de9c1878c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:01.553619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1b8ce-35dc-4116-9930-d2ff4565f81c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:01.554688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2c104-5d0d-4b0c-81d1-0f7de9c1878c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:01.558683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade894bf-290c-471a-8e65-3dcbe1b6719a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:01.559324 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade894bf-290c-471a-8e65-3dcbe1b6719a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.559606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade894bf-290c-471a-8e65-3dcbe1b6719a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.559879 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade894bf-290c-471a-8e65-3dcbe1b6719a admin admin] PATH_INFO: `/v3/projects/099d9268f9184ab28e23e9246f07c3be` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.561044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade894bf-290c-471a-8e65-3dcbe1b6719a admin admin] RBAC: Authorizing `identity:delete_project(project_id=099d9268f9184ab28e23e9246f07c3be)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:01.563003 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade894bf-290c-471a-8e65-3dcbe1b6719a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:01.565537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1057/2117] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:01.569742 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2c104-5d0d-4b0c-81d1-0f7de9c1878c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:01.581228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2c104-5d0d-4b0c-81d1-0f7de9c1878c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:01.586273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3edb4-d3a8-4914-b695-7c32abeafd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:01.586863 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3edb4-d3a8-4914-b695-7c32abeafd08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.587144 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3edb4-d3a8-4914-b695-7c32abeafd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.587416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3edb4-d3a8-4914-b695-7c32abeafd08 admin admin] PATH_INFO: `/v3/projects/b0b7b50254344bd3ab120e546cc15b39/users/d6f37c6e1a104e4fa3409f674a4f52c3/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.589624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3edb4-d3a8-4914-b695-7c32abeafd08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b7b50254344bd3ab120e546cc15b39, user_id=d6f37c6e1a104e4fa3409f674a4f52c3, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:01.592360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3edb4-d3a8-4914-b695-7c32abeafd08 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:01.606341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1058/2118] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:01 2022] PUT /identity/v3/projects/b0b7b50254344bd3ab120e546cc15b39/users/d6f37c6e1a104e4fa3409f674a4f52c3/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:16:01.609523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:01.615482 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ade894bf-290c-471a-8e65-3dcbe1b6719a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '099d9268f9184ab28e23e9246f07c3be', 'request_id': 'req-ade894bf-290c-471a-8e65-3dcbe1b6719a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:01.616554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024723ce-bb23-4f19-9f6e-e050a94c84d6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.616915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024723ce-bb23-4f19-9f6e-e050a94c84d6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.617212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024723ce-bb23-4f19-9f6e-e050a94c84d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.623818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ade894bf-290c-471a-8e65-3dcbe1b6719a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '099d9268f9184ab28e23e9246f07c3be', 'request_id': 'req-ade894bf-290c-471a-8e65-3dcbe1b6719a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:01.630815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ade894bf-290c-471a-8e65-3dcbe1b6719a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '099d9268f9184ab28e23e9246f07c3be', 'request_id': 'req-ade894bf-290c-471a-8e65-3dcbe1b6719a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:01.646857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ade894bf-290c-471a-8e65-3dcbe1b6719a admin admin] The token cache is being invalidate because project 099d9268f9184ab28e23e9246f07c3be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:01.647145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ade894bf-290c-471a-8e65-3dcbe1b6719a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:01.647676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ade894bf-290c-471a-8e65-3dcbe1b6719a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:01.650080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1061/2119] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:01 2022] DELETE /identity/v3/projects/099d9268f9184ab28e23e9246f07c3be => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:01.655920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239dc95-fd44-4fd8-8e60-62064992c1fb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.656233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239dc95-fd44-4fd8-8e60-62064992c1fb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.656509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239dc95-fd44-4fd8-8e60-62064992c1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.657700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-024723ce-bb23-4f19-9f6e-e050a94c84d6 None None] MFA Rules not processed for user `d6f37c6e1a104e4fa3409f674a4f52c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:01.659046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-024723ce-bb23-4f19-9f6e-e050a94c84d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:01.676989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2239dc95-fd44-4fd8-8e60-62064992c1fb None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:01.678212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2239dc95-fd44-4fd8-8e60-62064992c1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:01.702535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:01.737805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1059/2120] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:01.749326 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024723ce-bb23-4f19-9f6e-e050a94c84d6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:01.749700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1062/2121] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:01.750690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-024723ce-bb23-4f19-9f6e-e050a94c84d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:01.757505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53d8b7-ee4d-46fe-937c-522c382352b7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.757804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53d8b7-ee4d-46fe-937c-522c382352b7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.758194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53d8b7-ee4d-46fe-937c-522c382352b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.779076 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1c53d8b7-ee4d-46fe-937c-522c382352b7 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:01.780416 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1c53d8b7-ee4d-46fe-937c-522c382352b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:01.794236 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1063/2122] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:01.796082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53d8b7-ee4d-46fe-937c-522c382352b7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:01.807897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53d8b7-ee4d-46fe-937c-522c382352b7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:01.817791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024723ce-bb23-4f19-9f6e-e050a94c84d6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:01.819235 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37182f05-4060-4b12-9f13-c12ba1bf0f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:01.820045 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37182f05-4060-4b12-9f13-c12ba1bf0f8e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.820389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37182f05-4060-4b12-9f13-c12ba1bf0f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.820765 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37182f05-4060-4b12-9f13-c12ba1bf0f8e admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.821343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37182f05-4060-4b12-9f13-c12ba1bf0f8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:16:01.821648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37182f05-4060-4b12-9f13-c12ba1bf0f8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:01.821996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f74bd1-e461-4bba-90d4-458e27229bd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:01.822674 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f74bd1-e461-4bba-90d4-458e27229bd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.823301 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f74bd1-e461-4bba-90d4-458e27229bd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.823404 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37182f05-4060-4b12-9f13-c12ba1bf0f8e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:01.823502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f74bd1-e461-4bba-90d4-458e27229bd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.825460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b2f74bd1-e461-4bba-90d4-458e27229bd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:01.837807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1064/2123] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:16:01.842251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37182f05-4060-4b12-9f13-c12ba1bf0f8e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:01.852700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37182f05-4060-4b12-9f13-c12ba1bf0f8e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:01.857853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da581f1-23c0-4256-b0df-b9ddbc981742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:01.858452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da581f1-23c0-4256-b0df-b9ddbc981742 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.858724 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da581f1-23c0-4256-b0df-b9ddbc981742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.858990 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da581f1-23c0-4256-b0df-b9ddbc981742 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.859647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da581f1-23c0-4256-b0df-b9ddbc981742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:01.861587 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da581f1-23c0-4256-b0df-b9ddbc981742 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:01.892782 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f74bd1-e461-4bba-90d4-458e27229bd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:01.894394 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1065/2124] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:01 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:16:01.894725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f74bd1-e461-4bba-90d4-458e27229bd0 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:01.899315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da581f1-23c0-4256-b0df-b9ddbc981742 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:01.908058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1060/2125] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:16:01.966684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da581f1-23c0-4256-b0df-b9ddbc981742 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:01.971527 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50cd03-6027-4144-9435-bf2025d8750e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:01.972146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50cd03-6027-4144-9435-bf2025d8750e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:01.972425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50cd03-6027-4144-9435-bf2025d8750e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:01.972722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50cd03-6027-4144-9435-bf2025d8750e admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:01.973390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50cd03-6027-4144-9435-bf2025d8750e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:01.975393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50cd03-6027-4144-9435-bf2025d8750e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:02.001012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1066/2126] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:01 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:02.005256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f74bd1-e461-4bba-90d4-458e27229bd0 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:02.017267 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f74bd1-e461-4bba-90d4-458e27229bd0 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:02.026101 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d9d63-6411-4cf1-b1a9-4315fc7c7182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:02.026770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d9d63-6411-4cf1-b1a9-4315fc7c7182 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:02.027025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d9d63-6411-4cf1-b1a9-4315fc7c7182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:02.027441 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d9d63-6411-4cf1-b1a9-4315fc7c7182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:02.028112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d9d63-6411-4cf1-b1a9-4315fc7c7182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:02.029802 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d9d63-6411-4cf1-b1a9-4315fc7c7182 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:02.042449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1061/2127] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:02.046769 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50cd03-6027-4144-9435-bf2025d8750e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:02.058583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50cd03-6027-4144-9435-bf2025d8750e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:02.065709 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e0e46-a3c0-4aa0-abd7-8da10823f0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:02.066748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e0e46-a3c0-4aa0-abd7-8da10823f0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:02.066748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e0e46-a3c0-4aa0-abd7-8da10823f0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:02.066938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e0e46-a3c0-4aa0-abd7-8da10823f0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:02.068049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e0e46-a3c0-4aa0-abd7-8da10823f0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:02.069793 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e0e46-a3c0-4aa0-abd7-8da10823f0f9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:02.087380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1067/2128] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:02.092016 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d9d63-6411-4cf1-b1a9-4315fc7c7182 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:02.104147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d9d63-6411-4cf1-b1a9-4315fc7c7182 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:02.109206 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fe97f-7b4c-4540-99e5-57a5ae9b31ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:02.109877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fe97f-7b4c-4540-99e5-57a5ae9b31ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:02.110155 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fe97f-7b4c-4540-99e5-57a5ae9b31ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:02.110425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fe97f-7b4c-4540-99e5-57a5ae9b31ed admin admin] PATH_INFO: `/v3/projects/183480f0338b40ccb773b94a55d504af/users/20229315384e45ca83b397307c593708/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:02.127889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951fe97f-7b4c-4540-99e5-57a5ae9b31ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=183480f0338b40ccb773b94a55d504af, user_id=20229315384e45ca83b397307c593708, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:02.130611 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951fe97f-7b4c-4540-99e5-57a5ae9b31ed admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:02.146935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1062/2129] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:02 2022] PUT /identity/v3/projects/183480f0338b40ccb773b94a55d504af/users/20229315384e45ca83b397307c593708/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:02.157756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9af0cd-e425-4ca9-a54f-d782834bfbad None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:02.159130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9af0cd-e425-4ca9-a54f-d782834bfbad None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:02.159579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9af0cd-e425-4ca9-a54f-d782834bfbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:02.181130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:02.198454 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ef9af0cd-e425-4ca9-a54f-d782834bfbad None None] MFA Rules not processed for user `20229315384e45ca83b397307c593708`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:02.214105 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ef9af0cd-e425-4ca9-a54f-d782834bfbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:02.309721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1068/2130] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:02.314960 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fe97f-7b4c-4540-99e5-57a5ae9b31ed admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:02.316697 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-951fe97f-7b4c-4540-99e5-57a5ae9b31ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:02.373523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fe97f-7b4c-4540-99e5-57a5ae9b31ed admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:02.377335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20230ffe-3fb6-4c81-8824-fc4dbb1f033e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a76937ab52e445c59b9eb9638fd4a753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:02.377939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20230ffe-3fb6-4c81-8824-fc4dbb1f033e service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:02.378217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20230ffe-3fb6-4c81-8824-fc4dbb1f033e service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:02.378471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20230ffe-3fb6-4c81-8824-fc4dbb1f033e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:02.383950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:02.387678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20230ffe-3fb6-4c81-8824-fc4dbb1f033e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:02.389823 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20230ffe-3fb6-4c81-8824-fc4dbb1f033e service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:02.400165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1063/2131] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:05.654816 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9af0cd-e425-4ca9-a54f-d782834bfbad None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:05.745533 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9af0cd-e425-4ca9-a54f-d782834bfbad None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:05.751296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27191feb-55c3-4f3c-8cc8-0c55e89c1191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:05.752783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27191feb-55c3-4f3c-8cc8-0c55e89c1191 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:05.753295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27191feb-55c3-4f3c-8cc8-0c55e89c1191 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:05.754398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27191feb-55c3-4f3c-8cc8-0c55e89c1191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:05.757270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-27191feb-55c3-4f3c-8cc8-0c55e89c1191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:05.826197 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27191feb-55c3-4f3c-8cc8-0c55e89c1191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:05.827750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27191feb-55c3-4f3c-8cc8-0c55e89c1191 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:05.836911 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1069/2132] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:16:06.495219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20230ffe-3fb6-4c81-8824-fc4dbb1f033e service glance] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:06.496827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-20230ffe-3fb6-4c81-8824-fc4dbb1f033e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:06.510343 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20230ffe-3fb6-4c81-8824-fc4dbb1f033e service glance] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:06.514264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d337dd6-23d1-45bf-b37e-c463c7114bde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:06.514868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d337dd6-23d1-45bf-b37e-c463c7114bde service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:06.515139 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d337dd6-23d1-45bf-b37e-c463c7114bde service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:06.515401 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d337dd6-23d1-45bf-b37e-c463c7114bde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:06.580515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d337dd6-23d1-45bf-b37e-c463c7114bde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:06.581981 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d337dd6-23d1-45bf-b37e-c463c7114bde service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:06.594340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1064/2133] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:09.561503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27191feb-55c3-4f3c-8cc8-0c55e89c1191 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:09.563065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-27191feb-55c3-4f3c-8cc8-0c55e89c1191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:09.638975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:09.654574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27191feb-55c3-4f3c-8cc8-0c55e89c1191 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:09.659814 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:09.660533 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:09.660975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:09.661244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] PATH_INFO: `/v3/users/b784fd0a96994a04894fcc806c3f9968` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:09.662431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b784fd0a96994a04894fcc806c3f9968)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:09.664752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:09.727504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b784fd0a96994a04894fcc806c3f9968'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:09.728848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b784fd0a96994a04894fcc806c3f9968'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:09.745502 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b784fd0a96994a04894fcc806c3f9968'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:09.758601 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b784fd0a96994a04894fcc806c3f9968'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:09.759999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b784fd0a96994a04894fcc806c3f9968'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:09.773065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1070/2134] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:09 2022] DELETE /identity/v3/users/b784fd0a96994a04894fcc806c3f9968 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:09.777664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd2a7a-2647-456f-937e-370b10a41cb8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:09.778150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd2a7a-2647-456f-937e-370b10a41cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:09.778251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd2a7a-2647-456f-937e-370b10a41cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:09.811309 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b5cd2a7a-2647-456f-937e-370b10a41cb8 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:09.812689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b5cd2a7a-2647-456f-937e-370b10a41cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:09.903475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1065/2135] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:09.917316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:09.921071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:10.009393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7cba0-f2b8-46c7-8436-d7c208885bca admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:10.013698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cb2d0-1e79-4e21-a70d-17b1fb7de36b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:10.014674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cb2d0-1e79-4e21-a70d-17b1fb7de36b service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:10.014674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cb2d0-1e79-4e21-a70d-17b1fb7de36b service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:10.015507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cb2d0-1e79-4e21-a70d-17b1fb7de36b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:10.030025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85cb2d0-1e79-4e21-a70d-17b1fb7de36b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:10.031715 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85cb2d0-1e79-4e21-a70d-17b1fb7de36b service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:10.056972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1071/2136] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:10.591668 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd2a7a-2647-456f-937e-370b10a41cb8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:10.593368 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b5cd2a7a-2647-456f-937e-370b10a41cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:10.612875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd2a7a-2647-456f-937e-370b10a41cb8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:10.619488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:10.620142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:10.620598 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:10.620908 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] PATH_INFO: `/v3/users/c914e6a41ccc48f49194d57551bdf0d4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:10.621568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cb2d0-1e79-4e21-a70d-17b1fb7de36b service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:10.622070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c914e6a41ccc48f49194d57551bdf0d4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:10.623843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f85cb2d0-1e79-4e21-a70d-17b1fb7de36b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:10.624392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:10.639637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cb2d0-1e79-4e21-a70d-17b1fb7de36b service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:10.648815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996ed41-ff70-4d42-ae7a-1efad0733577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:10.648815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996ed41-ff70-4d42-ae7a-1efad0733577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:10.648815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996ed41-ff70-4d42-ae7a-1efad0733577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:10.648815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996ed41-ff70-4d42-ae7a-1efad0733577 admin admin] PATH_INFO: `/v3/projects/04f17cdc53b3464da70821f8926086a6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:10.652109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996ed41-ff70-4d42-ae7a-1efad0733577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f17cdc53b3464da70821f8926086a6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:10.654349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996ed41-ff70-4d42-ae7a-1efad0733577 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:10.688371 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c914e6a41ccc48f49194d57551bdf0d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.689492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c914e6a41ccc48f49194d57551bdf0d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.698386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c914e6a41ccc48f49194d57551bdf0d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.711805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c914e6a41ccc48f49194d57551bdf0d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.712552 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c914e6a41ccc48f49194d57551bdf0d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.719511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5996ed41-ff70-4d42-ae7a-1efad0733577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f17cdc53b3464da70821f8926086a6', 'request_id': 'req-5996ed41-ff70-4d42-ae7a-1efad0733577'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.727756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5996ed41-ff70-4d42-ae7a-1efad0733577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f17cdc53b3464da70821f8926086a6', 'request_id': 'req-5996ed41-ff70-4d42-ae7a-1efad0733577'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.736893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:10.738876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5996ed41-ff70-4d42-ae7a-1efad0733577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f17cdc53b3464da70821f8926086a6', 'request_id': 'req-5996ed41-ff70-4d42-ae7a-1efad0733577'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.743841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:10.744277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1066/2137] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:10 2022] DELETE /identity/v3/users/c914e6a41ccc48f49194d57551bdf0d4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:10.763473 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:10.765413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:10.766017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5996ed41-ff70-4d42-ae7a-1efad0733577 admin admin] The token cache is being invalidate because project 04f17cdc53b3464da70821f8926086a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:10.767055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5996ed41-ff70-4d42-ae7a-1efad0733577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.767629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5996ed41-ff70-4d42-ae7a-1efad0733577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.769712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1072/2138] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:10 2022] DELETE /identity/v3/projects/04f17cdc53b3464da70821f8926086a6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:10.774739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ae4c4-b451-46d2-979e-6c0677c8953f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:10.775022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ae4c4-b451-46d2-979e-6c0677c8953f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:10.775307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ae4c4-b451-46d2-979e-6c0677c8953f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:10.798567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8e5ae4c4-b451-46d2-979e-6c0677c8953f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:10.799656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8e5ae4c4-b451-46d2-979e-6c0677c8953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:10.849213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bccae-c976-4227-9012-5d64f6b698a1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:10.855127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b4b32-5369-44c1-bef0-87c257144a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:10.855568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b4b32-5369-44c1-bef0-87c257144a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:10.855955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b4b32-5369-44c1-bef0-87c257144a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:10.856958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b4b32-5369-44c1-bef0-87c257144a9f admin admin] PATH_INFO: `/v3/users/5982975fb70d47eca8ae9f148f81a2a5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:10.858705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b4b32-5369-44c1-bef0-87c257144a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5982975fb70d47eca8ae9f148f81a2a5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:10.860916 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b4b32-5369-44c1-bef0-87c257144a9f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:10.883889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1073/2139] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:10.891582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd780b-995c-4a64-b962-ae03675dfb14 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:10.891868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd780b-995c-4a64-b962-ae03675dfb14 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:10.892141 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd780b-995c-4a64-b962-ae03675dfb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:10.929671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fffd780b-995c-4a64-b962-ae03675dfb14 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:10.931002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fffd780b-995c-4a64-b962-ae03675dfb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:10.937738 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-478b4b32-5369-44c1-bef0-87c257144a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5982975fb70d47eca8ae9f148f81a2a5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.938657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-478b4b32-5369-44c1-bef0-87c257144a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5982975fb70d47eca8ae9f148f81a2a5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.939786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1074/2140] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:10.944061 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd780b-995c-4a64-b962-ae03675dfb14 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:10.947860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-478b4b32-5369-44c1-bef0-87c257144a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5982975fb70d47eca8ae9f148f81a2a5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.955350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd780b-995c-4a64-b962-ae03675dfb14 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:10.961238 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-478b4b32-5369-44c1-bef0-87c257144a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5982975fb70d47eca8ae9f148f81a2a5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.961678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc74fa-4566-4168-9614-2cb27937a148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:10.961959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-478b4b32-5369-44c1-bef0-87c257144a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5982975fb70d47eca8ae9f148f81a2a5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:10.962303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc74fa-4566-4168-9614-2cb27937a148 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:10.962586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc74fa-4566-4168-9614-2cb27937a148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:10.962855 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc74fa-4566-4168-9614-2cb27937a148 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:10.963604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc74fa-4566-4168-9614-2cb27937a148 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:16:10.963802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc74fa-4566-4168-9614-2cb27937a148 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:10.965500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc74fa-4566-4168-9614-2cb27937a148 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:10.986182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1075/2141] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:16:10.989822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1067/2142] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:10 2022] DELETE /identity/v3/users/5982975fb70d47eca8ae9f148f81a2a5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:16:10.990849 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc74fa-4566-4168-9614-2cb27937a148 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:10.994272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9a70d-7626-4236-908e-38c8bb0f0b26 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:10.994751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9a70d-7626-4236-908e-38c8bb0f0b26 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:10.994866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-22bc74fa-4566-4168-9614-2cb27937a148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:10.995268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9a70d-7626-4236-908e-38c8bb0f0b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.012698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc74fa-4566-4168-9614-2cb27937a148 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.020672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-80c9a70d-7626-4236-908e-38c8bb0f0b26 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:11.021942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-80c9a70d-7626-4236-908e-38c8bb0f0b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:11.090058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8d173-a399-4750-9134-31a348180ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.090687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8d173-a399-4750-9134-31a348180ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.090996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8d173-a399-4750-9134-31a348180ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.091347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8d173-a399-4750-9134-31a348180ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.091987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8d173-a399-4750-9134-31a348180ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.093951 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8d173-a399-4750-9134-31a348180ab1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.113800 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1068/2143] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:11.114837 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9a70d-7626-4236-908e-38c8bb0f0b26 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.116263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-80c9a70d-7626-4236-908e-38c8bb0f0b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:11.130370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1076/2144] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:10 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:11.130370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8d173-a399-4750-9134-31a348180ab1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.130666 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9a70d-7626-4236-908e-38c8bb0f0b26 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.131903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-02e8d173-a399-4750-9134-31a348180ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:11.145488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:11.197496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369a354-56e8-4f39-84e8-9621ee4c0af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.198154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369a354-56e8-4f39-84e8-9621ee4c0af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.198427 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369a354-56e8-4f39-84e8-9621ee4c0af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.198700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369a354-56e8-4f39-84e8-9621ee4c0af6 admin admin] PATH_INFO: `/v3/projects/a2ec2d845bf247a6b463e59e513dd936` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.199858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369a354-56e8-4f39-84e8-9621ee4c0af6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ec2d845bf247a6b463e59e513dd936)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.200503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8d173-a399-4750-9134-31a348180ab1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.201984 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369a354-56e8-4f39-84e8-9621ee4c0af6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.207426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3edfc-e45b-4929-bbc4-60e082cacd90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.208438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3edfc-e45b-4929-bbc4-60e082cacd90 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.208960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3edfc-e45b-4929-bbc4-60e082cacd90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.210046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3edfc-e45b-4929-bbc4-60e082cacd90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.221958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3edfc-e45b-4929-bbc4-60e082cacd90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.223449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3edfc-e45b-4929-bbc4-60e082cacd90 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.233950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1077/2145] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:11.234874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3edfc-e45b-4929-bbc4-60e082cacd90 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.241001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:11.246695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3edfc-e45b-4929-bbc4-60e082cacd90 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.260827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6369a354-56e8-4f39-84e8-9621ee4c0af6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ec2d845bf247a6b463e59e513dd936', 'request_id': 'req-6369a354-56e8-4f39-84e8-9621ee4c0af6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.270578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6369a354-56e8-4f39-84e8-9621ee4c0af6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ec2d845bf247a6b463e59e513dd936', 'request_id': 'req-6369a354-56e8-4f39-84e8-9621ee4c0af6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.279530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6369a354-56e8-4f39-84e8-9621ee4c0af6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ec2d845bf247a6b463e59e513dd936', 'request_id': 'req-6369a354-56e8-4f39-84e8-9621ee4c0af6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.294426 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6369a354-56e8-4f39-84e8-9621ee4c0af6 admin admin] The token cache is being invalidate because project a2ec2d845bf247a6b463e59e513dd936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:11.294766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6369a354-56e8-4f39-84e8-9621ee4c0af6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.295454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6369a354-56e8-4f39-84e8-9621ee4c0af6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.297357 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1069/2146] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:11 2022] DELETE /identity/v3/projects/a2ec2d845bf247a6b463e59e513dd936 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:11.302759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13105e03-e695-4a55-99d2-83587d85eb46 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.302910 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13105e03-e695-4a55-99d2-83587d85eb46 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.303070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13105e03-e695-4a55-99d2-83587d85eb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.334128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-13105e03-e695-4a55-99d2-83587d85eb46 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:11.335428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2de801-eaa8-498b-b105-555017cd8e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.335570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-13105e03-e695-4a55-99d2-83587d85eb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:11.335855 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2de801-eaa8-498b-b105-555017cd8e21 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.336130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2de801-eaa8-498b-b105-555017cd8e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.336566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2de801-eaa8-498b-b105-555017cd8e21 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.337240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2de801-eaa8-498b-b105-555017cd8e21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.339167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2de801-eaa8-498b-b105-555017cd8e21 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.351543 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1070/2147] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:11.355056 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006d4b1-b187-412e-aa2f-39ac74b85457 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.355344 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006d4b1-b187-412e-aa2f-39ac74b85457 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.355621 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006d4b1-b187-412e-aa2f-39ac74b85457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.378544 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1078/2148] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:11 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:11.382705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1006d4b1-b187-412e-aa2f-39ac74b85457 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:11.383749 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2de801-eaa8-498b-b105-555017cd8e21 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.385936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1006d4b1-b187-412e-aa2f-39ac74b85457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:11.388617 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7b2de801-eaa8-498b-b105-555017cd8e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:11.397493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1071/2149] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:11.403952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006d4b1-b187-412e-aa2f-39ac74b85457 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.407553 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2de801-eaa8-498b-b105-555017cd8e21 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.429532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa220f3-b267-4161-8924-e48f054a4dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.430988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa220f3-b267-4161-8924-e48f054a4dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.431578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa220f3-b267-4161-8924-e48f054a4dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.432161 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa220f3-b267-4161-8924-e48f054a4dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.432920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa220f3-b267-4161-8924-e48f054a4dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.435151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa220f3-b267-4161-8924-e48f054a4dc1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.450647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1079/2150] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:16:11.452945 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006d4b1-b187-412e-aa2f-39ac74b85457 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.454740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa220f3-b267-4161-8924-e48f054a4dc1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.459777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8a943-5ccb-4fcf-8bf4-6e3677eabb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.460409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8a943-5ccb-4fcf-8bf4-6e3677eabb12 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.460721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8a943-5ccb-4fcf-8bf4-6e3677eabb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.461000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8a943-5ccb-4fcf-8bf4-6e3677eabb12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.461824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8a943-5ccb-4fcf-8bf4-6e3677eabb12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:16:11.462129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8a943-5ccb-4fcf-8bf4-6e3677eabb12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.464469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8a943-5ccb-4fcf-8bf4-6e3677eabb12 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.476498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa220f3-b267-4161-8924-e48f054a4dc1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.480711 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1072/2151] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:16:11.485373 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8a943-5ccb-4fcf-8bf4-6e3677eabb12 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.492531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f72bd-aa30-45a8-ab06-d6a9336917f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.493364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f72bd-aa30-45a8-ab06-d6a9336917f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.493728 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f72bd-aa30-45a8-ab06-d6a9336917f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.494525 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f72bd-aa30-45a8-ab06-d6a9336917f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.495189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f72bd-aa30-45a8-ab06-d6a9336917f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.499904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8a943-5ccb-4fcf-8bf4-6e3677eabb12 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.500090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f72bd-aa30-45a8-ab06-d6a9336917f5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.505304 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87619d8-049d-4979-bf02-259ab83781e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.506226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87619d8-049d-4979-bf02-259ab83781e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.506516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87619d8-049d-4979-bf02-259ab83781e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.506803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87619d8-049d-4979-bf02-259ab83781e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.507436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87619d8-049d-4979-bf02-259ab83781e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.509402 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87619d8-049d-4979-bf02-259ab83781e8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.511272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1080/2152] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:11.516017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f72bd-aa30-45a8-ab06-d6a9336917f5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.526280 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f72bd-aa30-45a8-ab06-d6a9336917f5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.532200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe0d2b-4ca6-40c3-9847-02a1894b4c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.532948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe0d2b-4ca6-40c3-9847-02a1894b4c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.533232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe0d2b-4ca6-40c3-9847-02a1894b4c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.533872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe0d2b-4ca6-40c3-9847-02a1894b4c72 admin admin] PATH_INFO: `/v3/projects/4d2fc10c7eef4dfe89030d88e3378ccc/users/09830ff3c16543e19a1c95791aa0dc76/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.544697 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1073/2153] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:11 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:16:11.549739 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87619d8-049d-4979-bf02-259ab83781e8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.555610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe0d2b-4ca6-40c3-9847-02a1894b4c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d2fc10c7eef4dfe89030d88e3378ccc, user_id=09830ff3c16543e19a1c95791aa0dc76, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.560922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe0d2b-4ca6-40c3-9847-02a1894b4c72 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.593151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1081/2154] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:11 2022] PUT /identity/v3/projects/4d2fc10c7eef4dfe89030d88e3378ccc/users/09830ff3c16543e19a1c95791aa0dc76/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:16:11.594119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe0d2b-4ca6-40c3-9847-02a1894b4c72 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.596414 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8dbe0d2b-4ca6-40c3-9847-02a1894b4c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:11.638943 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87619d8-049d-4979-bf02-259ab83781e8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.644048 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:11.668453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe0d2b-4ca6-40c3-9847-02a1894b4c72 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.673289 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.674329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.674707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.675055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] PATH_INFO: `/v3/projects/375ebb311dfc4034b2565cbc16c18c2b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.676237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=375ebb311dfc4034b2565cbc16c18c2b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.678393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.710560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55d390-9ed0-4664-ab89-d4e669a0a618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.711186 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55d390-9ed0-4664-ab89-d4e669a0a618 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.711453 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55d390-9ed0-4664-ab89-d4e669a0a618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.711722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55d390-9ed0-4664-ab89-d4e669a0a618 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.712352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55d390-9ed0-4664-ab89-d4e669a0a618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.714434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55d390-9ed0-4664-ab89-d4e669a0a618 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.726933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375ebb311dfc4034b2565cbc16c18c2b', 'request_id': 'req-ff1af38b-882c-483a-b53b-3cfe406c45c6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.727892 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:11.735458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375ebb311dfc4034b2565cbc16c18c2b', 'request_id': 'req-ff1af38b-882c-483a-b53b-3cfe406c45c6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.741407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1074/2155] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:11 2022] POST /identity/v3/users => generated 284 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:11.743193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f9ef2-a2bc-41a0-89d9-45ac21c54a06 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.743461 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f9ef2-a2bc-41a0-89d9-45ac21c54a06 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.743734 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f9ef2-a2bc-41a0-89d9-45ac21c54a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.746037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375ebb311dfc4034b2565cbc16c18c2b', 'request_id': 'req-ff1af38b-882c-483a-b53b-3cfe406c45c6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.758525 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] The token cache is being invalidate because project 375ebb311dfc4034b2565cbc16c18c2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:11.758813 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.759440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.761478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1082/2156] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:11 2022] DELETE /identity/v3/projects/375ebb311dfc4034b2565cbc16c18c2b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:11.762256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.763720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:11.778886 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-068f9ef2-a2bc-41a0-89d9-45ac21c54a06 None None] MFA Rules not processed for user `09830ff3c16543e19a1c95791aa0dc76`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:11.780119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-068f9ef2-a2bc-41a0-89d9-45ac21c54a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:11.830509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1af38b-882c-483a-b53b-3cfe406c45c6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.836775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7263de-fcec-48e0-baeb-1f3ef96559d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.837437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7263de-fcec-48e0-baeb-1f3ef96559d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.837775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7263de-fcec-48e0-baeb-1f3ef96559d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.838120 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7263de-fcec-48e0-baeb-1f3ef96559d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.838776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7263de-fcec-48e0-baeb-1f3ef96559d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.840644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7263de-fcec-48e0-baeb-1f3ef96559d5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.851676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1083/2157] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:11.852612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7263de-fcec-48e0-baeb-1f3ef96559d5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.857699 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1075/2158] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:11.857699 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f9ef2-a2bc-41a0-89d9-45ac21c54a06 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.859121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9f7263de-fcec-48e0-baeb-1f3ef96559d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:11.872827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f9ef2-a2bc-41a0-89d9-45ac21c54a06 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.874034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7263de-fcec-48e0-baeb-1f3ef96559d5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.877620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d5943-718e-4086-9190-b8eafb5a3118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.878265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d5943-718e-4086-9190-b8eafb5a3118 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.878542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d5943-718e-4086-9190-b8eafb5a3118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.878639 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.878815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d5943-718e-4086-9190-b8eafb5a3118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.879223 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.879463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d5943-718e-4086-9190-b8eafb5a3118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.879570 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.879750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] PATH_INFO: `/v3/users/27551682c7904696a057504c52272a6c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.880888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] RBAC: Authorizing `identity:delete_user(user_id=27551682c7904696a057504c52272a6c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.881118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d5943-718e-4086-9190-b8eafb5a3118 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.882811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:11.891832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1076/2159] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:16:11.892612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d5943-718e-4086-9190-b8eafb5a3118 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.896252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-484d5943-718e-4086-9190-b8eafb5a3118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:11.939511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27551682c7904696a057504c52272a6c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.940358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27551682c7904696a057504c52272a6c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.949862 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27551682c7904696a057504c52272a6c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.962446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27551682c7904696a057504c52272a6c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.963156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27551682c7904696a057504c52272a6c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:11.969919 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d5943-718e-4086-9190-b8eafb5a3118 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:11.973813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de716f-3a1f-4aca-a5a5-526b4339ab3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:11.974429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de716f-3a1f-4aca-a5a5-526b4339ab3b service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:11.974705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de716f-3a1f-4aca-a5a5-526b4339ab3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:11.974974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de716f-3a1f-4aca-a5a5-526b4339ab3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:11.976591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1084/2160] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:11 2022] DELETE /identity/v3/users/27551682c7904696a057504c52272a6c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:11.976821 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-76de716f-3a1f-4aca-a5a5-526b4339ab3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:11.977589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:11.978959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:11.988502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de716f-3a1f-4aca-a5a5-526b4339ab3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:11.989989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de716f-3a1f-4aca-a5a5-526b4339ab3b service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:12.001034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1077/2161] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:12.036057 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:12.061257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2121c26-b69a-4b94-bbb0-6adddc43261a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:12.074102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0565708-e15d-427a-92f3-ab9ae6d3cee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:12.075597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0565708-e15d-427a-92f3-ab9ae6d3cee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:12.075597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0565708-e15d-427a-92f3-ab9ae6d3cee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:12.076022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0565708-e15d-427a-92f3-ab9ae6d3cee2 admin admin] PATH_INFO: `/v3/projects/e93a4e19a05f400687773aee4dbd64e3/users/854cd4dd926c45519c3e32ac7e7c65a2/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:12.109558 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0565708-e15d-427a-92f3-ab9ae6d3cee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93a4e19a05f400687773aee4dbd64e3, user_id=854cd4dd926c45519c3e32ac7e7c65a2, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:12.112921 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0565708-e15d-427a-92f3-ab9ae6d3cee2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:12.131997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1085/2162] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:11 2022] PUT /identity/v3/projects/e93a4e19a05f400687773aee4dbd64e3/users/854cd4dd926c45519c3e32ac7e7c65a2/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:12.148337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c746edb-b159-4a5d-af80-dc2de3d14254 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:12.149033 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c746edb-b159-4a5d-af80-dc2de3d14254 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:12.149835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c746edb-b159-4a5d-af80-dc2de3d14254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:12.198524 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9c746edb-b159-4a5d-af80-dc2de3d14254 None None] MFA Rules not processed for user `854cd4dd926c45519c3e32ac7e7c65a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:12.209976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9c746edb-b159-4a5d-af80-dc2de3d14254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:12.233037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:12.334624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1078/2163] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:12.346863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0565708-e15d-427a-92f3-ab9ae6d3cee2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:12.347042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c746edb-b159-4a5d-af80-dc2de3d14254 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:12.348471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9c746edb-b159-4a5d-af80-dc2de3d14254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:12.349239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e0565708-e15d-427a-92f3-ab9ae6d3cee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:12.458155 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0565708-e15d-427a-92f3-ab9ae6d3cee2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:12.461484 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c746edb-b159-4a5d-af80-dc2de3d14254 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:12.463667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc479c-d81e-41ac-8bae-4f225a4b773a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:12.464947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc479c-d81e-41ac-8bae-4f225a4b773a service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:12.465284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc479c-d81e-41ac-8bae-4f225a4b773a service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:12.465563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc479c-d81e-41ac-8bae-4f225a4b773a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:12.468672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:12.469459 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:12.469731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:12.470113 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] PATH_INFO: `/v3/projects/65d63841670549a1b5cbcce285c7c172` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:12.471470 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] RBAC: Authorizing `identity:delete_project(project_id=65d63841670549a1b5cbcce285c7c172)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:12.474592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:12.477470 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc479c-d81e-41ac-8bae-4f225a4b773a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:12.483756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc479c-d81e-41ac-8bae-4f225a4b773a service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:12.510732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1086/2164] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:12.545329 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65d63841670549a1b5cbcce285c7c172', 'request_id': 'req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:12.565948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65d63841670549a1b5cbcce285c7c172', 'request_id': 'req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:12.571328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65d63841670549a1b5cbcce285c7c172', 'request_id': 'req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:12.586693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] The token cache is being invalidate because project 65d63841670549a1b5cbcce285c7c172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:12.586693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:12.586693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:12.587994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1079/2165] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:12 2022] DELETE /identity/v3/projects/65d63841670549a1b5cbcce285c7c172 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:12.592307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc479c-d81e-41ac-8bae-4f225a4b773a service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:12.610209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc479c-d81e-41ac-8bae-4f225a4b773a service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:12.672833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129b0b9-bb7b-4114-b39f-2e36a944dd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:12.673586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129b0b9-bb7b-4114-b39f-2e36a944dd23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:12.673748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129b0b9-bb7b-4114-b39f-2e36a944dd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:12.675092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129b0b9-bb7b-4114-b39f-2e36a944dd23 admin admin] PATH_INFO: `/v3/users/d6f37c6e1a104e4fa3409f674a4f52c3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:12.676275 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129b0b9-bb7b-4114-b39f-2e36a944dd23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6f37c6e1a104e4fa3409f674a4f52c3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:12.678271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129b0b9-bb7b-4114-b39f-2e36a944dd23 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:12.728501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d129b0b9-bb7b-4114-b39f-2e36a944dd23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6f37c6e1a104e4fa3409f674a4f52c3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:12.729239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d129b0b9-bb7b-4114-b39f-2e36a944dd23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6f37c6e1a104e4fa3409f674a4f52c3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:12.736745 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d129b0b9-bb7b-4114-b39f-2e36a944dd23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6f37c6e1a104e4fa3409f674a4f52c3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:12.749731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d129b0b9-bb7b-4114-b39f-2e36a944dd23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6f37c6e1a104e4fa3409f674a4f52c3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:12.750543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d129b0b9-bb7b-4114-b39f-2e36a944dd23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6f37c6e1a104e4fa3409f674a4f52c3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:12.752245 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:12.766793 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1087/2166] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:12 2022] DELETE /identity/v3/users/d6f37c6e1a104e4fa3409f674a4f52c3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:13.089281 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:13.090750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:13.108730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca0a4-c80f-4a4a-8a25-d73a3f0c353c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:13.155836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:13.177967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773153d9-1b32-441c-aa44-3b6e4d1859e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:13.178610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773153d9-1b32-441c-aa44-3b6e4d1859e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:13.178892 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773153d9-1b32-441c-aa44-3b6e4d1859e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:13.179163 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773153d9-1b32-441c-aa44-3b6e4d1859e4 admin admin] PATH_INFO: `/v3/projects/b0b7b50254344bd3ab120e546cc15b39` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:13.180283 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773153d9-1b32-441c-aa44-3b6e4d1859e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b7b50254344bd3ab120e546cc15b39)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:13.182257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773153d9-1b32-441c-aa44-3b6e4d1859e4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:13.234194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-773153d9-1b32-441c-aa44-3b6e4d1859e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b7b50254344bd3ab120e546cc15b39', 'request_id': 'req-773153d9-1b32-441c-aa44-3b6e4d1859e4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:13.244074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-773153d9-1b32-441c-aa44-3b6e4d1859e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b7b50254344bd3ab120e546cc15b39', 'request_id': 'req-773153d9-1b32-441c-aa44-3b6e4d1859e4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:13.251533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-773153d9-1b32-441c-aa44-3b6e4d1859e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b7b50254344bd3ab120e546cc15b39', 'request_id': 'req-773153d9-1b32-441c-aa44-3b6e4d1859e4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:13.263776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-773153d9-1b32-441c-aa44-3b6e4d1859e4 admin admin] The token cache is being invalidate because project b0b7b50254344bd3ab120e546cc15b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:13.264739 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-773153d9-1b32-441c-aa44-3b6e4d1859e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:13.265503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-773153d9-1b32-441c-aa44-3b6e4d1859e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:13.267059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1080/2167] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:13 2022] DELETE /identity/v3/projects/b0b7b50254344bd3ab120e546cc15b39 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:13.272190 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b24878-59ce-4361-a126-16012a48a848 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:13.272409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b24878-59ce-4361-a126-16012a48a848 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:13.272636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b24878-59ce-4361-a126-16012a48a848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:13.292675 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-38b24878-59ce-4361-a126-16012a48a848 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:13.293886 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-38b24878-59ce-4361-a126-16012a48a848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:13.370869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1088/2168] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:13.381504 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e5e75-2857-48cc-8317-55369764d345 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:13.381789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e5e75-2857-48cc-8317-55369764d345 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:13.382122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e5e75-2857-48cc-8317-55369764d345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:13.403937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e40e5e75-2857-48cc-8317-55369764d345 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:13.405238 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e40e5e75-2857-48cc-8317-55369764d345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:13.413931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1081/2169] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:13.419185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b24878-59ce-4361-a126-16012a48a848 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:13.430513 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b24878-59ce-4361-a126-16012a48a848 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:13.435493 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e2a4a-2c86-4b45-be66-f352bbb45f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:13.436230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e2a4a-2c86-4b45-be66-f352bbb45f37 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:13.436525 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e2a4a-2c86-4b45-be66-f352bbb45f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:13.437000 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e2a4a-2c86-4b45-be66-f352bbb45f37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:13.437962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e2a4a-2c86-4b45-be66-f352bbb45f37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:16:13.438291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e2a4a-2c86-4b45-be66-f352bbb45f37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:13.440330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e2a4a-2c86-4b45-be66-f352bbb45f37 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:13.456337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1089/2170] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:16:13.457678 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e5e75-2857-48cc-8317-55369764d345 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:13.468159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e5e75-2857-48cc-8317-55369764d345 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:13.473106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7decf-d265-440a-bf15-f928c48456e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:13.473716 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7decf-d265-440a-bf15-f928c48456e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:13.473989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7decf-d265-440a-bf15-f928c48456e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:13.474255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7decf-d265-440a-bf15-f928c48456e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:13.474884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7decf-d265-440a-bf15-f928c48456e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:13.476793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7decf-d265-440a-bf15-f928c48456e0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:13.519715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1082/2171] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:13 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:13.520884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e2a4a-2c86-4b45-be66-f352bbb45f37 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:13.531168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e2a4a-2c86-4b45-be66-f352bbb45f37 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:13.592408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319630d6-d3fd-49fd-83cb-59a8a4680ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:13.593113 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319630d6-d3fd-49fd-83cb-59a8a4680ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:13.593391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319630d6-d3fd-49fd-83cb-59a8a4680ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:13.593654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319630d6-d3fd-49fd-83cb-59a8a4680ce9 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:13.594360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319630d6-d3fd-49fd-83cb-59a8a4680ce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:13.596332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319630d6-d3fd-49fd-83cb-59a8a4680ce9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:13.622456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1090/2172] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:13 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:16:13.623714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7decf-d265-440a-bf15-f928c48456e0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:13.640783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7decf-d265-440a-bf15-f928c48456e0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:13.645685 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183210ca-efb7-4595-af4a-c58dbc8a8bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:13.646335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183210ca-efb7-4595-af4a-c58dbc8a8bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:13.646680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183210ca-efb7-4595-af4a-c58dbc8a8bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:13.646973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183210ca-efb7-4595-af4a-c58dbc8a8bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:13.647732 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183210ca-efb7-4595-af4a-c58dbc8a8bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:13.649494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183210ca-efb7-4595-af4a-c58dbc8a8bb2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:13.675797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1083/2173] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:16:13.680467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319630d6-d3fd-49fd-83cb-59a8a4680ce9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:13.691612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319630d6-d3fd-49fd-83cb-59a8a4680ce9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:13.696874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f389185-3c9e-4710-846b-485fc5de63c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:13.697500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f389185-3c9e-4710-846b-485fc5de63c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:13.697776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f389185-3c9e-4710-846b-485fc5de63c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:13.698201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f389185-3c9e-4710-846b-485fc5de63c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:13.698860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f389185-3c9e-4710-846b-485fc5de63c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:13.700534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f389185-3c9e-4710-846b-485fc5de63c0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:13.712696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1091/2174] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:13.717367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183210ca-efb7-4595-af4a-c58dbc8a8bb2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:13.737216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183210ca-efb7-4595-af4a-c58dbc8a8bb2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:13.740891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a7bc1-76c3-4efc-a1db-04379cda8049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:13.741673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a7bc1-76c3-4efc-a1db-04379cda8049 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:13.741954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a7bc1-76c3-4efc-a1db-04379cda8049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:13.742227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a7bc1-76c3-4efc-a1db-04379cda8049 admin admin] PATH_INFO: `/v3/projects/9972bb74beee4b17a831b300ad81e139/users/20cf45c85a844d548400e963dec5059c/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:13.759025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a7bc1-76c3-4efc-a1db-04379cda8049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9972bb74beee4b17a831b300ad81e139, user_id=20cf45c85a844d548400e963dec5059c, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:13.761899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a7bc1-76c3-4efc-a1db-04379cda8049 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:13.775443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1084/2175] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:13 2022] PUT /identity/v3/projects/9972bb74beee4b17a831b300ad81e139/users/20cf45c85a844d548400e963dec5059c/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:13.786325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde91799-b76e-408e-a8ae-6ad6632c3d6f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:13.786641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde91799-b76e-408e-a8ae-6ad6632c3d6f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:13.786924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde91799-b76e-408e-a8ae-6ad6632c3d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:13.817432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fde91799-b76e-408e-a8ae-6ad6632c3d6f None None] MFA Rules not processed for user `20cf45c85a844d548400e963dec5059c`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:13.818743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fde91799-b76e-408e-a8ae-6ad6632c3d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:13.877162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:13.900026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1092/2176] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:13.913651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a7bc1-76c3-4efc-a1db-04379cda8049 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:13.918328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-715a7bc1-76c3-4efc-a1db-04379cda8049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:13.978019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a7bc1-76c3-4efc-a1db-04379cda8049 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:13.982597 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf43d91-695e-4803-b6a5-5f44af4617ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:13.983216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf43d91-695e-4803-b6a5-5f44af4617ad service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:13.983491 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf43d91-695e-4803-b6a5-5f44af4617ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:13.983759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf43d91-695e-4803-b6a5-5f44af4617ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:13.993383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf43d91-695e-4803-b6a5-5f44af4617ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:13.994914 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf43d91-695e-4803-b6a5-5f44af4617ad service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:14.005503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1085/2177] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:14.035014 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde91799-b76e-408e-a8ae-6ad6632c3d6f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:14.102396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde91799-b76e-408e-a8ae-6ad6632c3d6f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:14.111009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:14.111903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:14.112253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:14.112608 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] PATH_INFO: `/v3/users/20cf45c85a844d548400e963dec5059c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:14.114473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20cf45c85a844d548400e963dec5059c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:14.117465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:14.170157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20cf45c85a844d548400e963dec5059c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:14.170842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20cf45c85a844d548400e963dec5059c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:14.178649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20cf45c85a844d548400e963dec5059c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:14.193127 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20cf45c85a844d548400e963dec5059c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:14.193815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20cf45c85a844d548400e963dec5059c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:14.211818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1093/2178] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:14 2022] DELETE /identity/v3/users/20cf45c85a844d548400e963dec5059c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:14.216904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68175799-88ef-4b28-b82e-28dcf1e84217 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:14.217384 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68175799-88ef-4b28-b82e-28dcf1e84217 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:14.217505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68175799-88ef-4b28-b82e-28dcf1e84217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:14.237561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-68175799-88ef-4b28-b82e-28dcf1e84217 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:14.238762 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-68175799-88ef-4b28-b82e-28dcf1e84217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:14.243037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:14.305158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1086/2179] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:14.315413 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:14.317004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:14.380953 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec16654-5761-4865-a173-d92751caffa8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:14.384613 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a510d-c981-4b08-bae8-96d3c8c27210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:14.385263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a510d-c981-4b08-bae8-96d3c8c27210 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:14.385539 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a510d-c981-4b08-bae8-96d3c8c27210 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:14.385809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a510d-c981-4b08-bae8-96d3c8c27210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:14.395132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a510d-c981-4b08-bae8-96d3c8c27210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:14.396597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a510d-c981-4b08-bae8-96d3c8c27210 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:14.407345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1094/2180] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:14.736947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68175799-88ef-4b28-b82e-28dcf1e84217 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:14.738384 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-68175799-88ef-4b28-b82e-28dcf1e84217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:14.759658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68175799-88ef-4b28-b82e-28dcf1e84217 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:14.766622 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:14.772766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:14.772766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:14.772766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] PATH_INFO: `/v3/projects/9972bb74beee4b17a831b300ad81e139` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:14.772766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9972bb74beee4b17a831b300ad81e139)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:14.772766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:14.830880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9972bb74beee4b17a831b300ad81e139', 'request_id': 'req-0f332041-52b7-4d15-819b-2600fd20b446'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:14.839156 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9972bb74beee4b17a831b300ad81e139', 'request_id': 'req-0f332041-52b7-4d15-819b-2600fd20b446'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:14.845792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9972bb74beee4b17a831b300ad81e139', 'request_id': 'req-0f332041-52b7-4d15-819b-2600fd20b446'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:14.857134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] The token cache is being invalidate because project 9972bb74beee4b17a831b300ad81e139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:14.857351 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:14.858003 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:14.859607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1087/2181] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:14 2022] DELETE /identity/v3/projects/9972bb74beee4b17a831b300ad81e139 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:14.864763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d7140-5080-4ff3-9e25-c68cdaeeb791 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:14.865237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d7140-5080-4ff3-9e25-c68cdaeeb791 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:14.865332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d7140-5080-4ff3-9e25-c68cdaeeb791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:14.868053 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:14.869433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:14.886125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5e3d7140-5080-4ff3-9e25-c68cdaeeb791 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:14.887472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5e3d7140-5080-4ff3-9e25-c68cdaeeb791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:14.892636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:14.953250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f332041-52b7-4d15-819b-2600fd20b446 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:14.958576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906d5e1-6a3e-4dc3-9e97-8dadf6277b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:14.959334 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906d5e1-6a3e-4dc3-9e97-8dadf6277b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:14.959776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906d5e1-6a3e-4dc3-9e97-8dadf6277b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:14.960070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906d5e1-6a3e-4dc3-9e97-8dadf6277b20 admin admin] PATH_INFO: `/v3/users/09830ff3c16543e19a1c95791aa0dc76` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:14.961469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906d5e1-6a3e-4dc3-9e97-8dadf6277b20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09830ff3c16543e19a1c95791aa0dc76)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:14.963754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906d5e1-6a3e-4dc3-9e97-8dadf6277b20 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:14.965419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1095/2182] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:14.973471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe5db9-9fbd-4ec4-a864-d53cea4b30f1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:14.973768 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe5db9-9fbd-4ec4-a864-d53cea4b30f1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:14.974050 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe5db9-9fbd-4ec4-a864-d53cea4b30f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:14.995036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-02fe5db9-9fbd-4ec4-a864-d53cea4b30f1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:14.996337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-02fe5db9-9fbd-4ec4-a864-d53cea4b30f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:15.004925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1096/2183] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:15.008847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe5db9-9fbd-4ec4-a864-d53cea4b30f1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:15.018690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe5db9-9fbd-4ec4-a864-d53cea4b30f1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:15.023953 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9b44f-11ba-4a3d-9b4d-6fdc997a7943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:15.024636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9b44f-11ba-4a3d-9b4d-6fdc997a7943 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.024973 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9b44f-11ba-4a3d-9b4d-6fdc997a7943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.025314 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9b44f-11ba-4a3d-9b4d-6fdc997a7943 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.026113 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9b44f-11ba-4a3d-9b4d-6fdc997a7943 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:16:15.026455 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9b44f-11ba-4a3d-9b4d-6fdc997a7943 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:15.028207 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9b44f-11ba-4a3d-9b4d-6fdc997a7943 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:15.036482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:15.041163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1097/2184] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:16:15.047578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9b44f-11ba-4a3d-9b4d-6fdc997a7943 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:15.055683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4906d5e1-6a3e-4dc3-9e97-8dadf6277b20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09830ff3c16543e19a1c95791aa0dc76'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:15.056815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4906d5e1-6a3e-4dc3-9e97-8dadf6277b20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09830ff3c16543e19a1c95791aa0dc76'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:15.059359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9b44f-11ba-4a3d-9b4d-6fdc997a7943 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:15.064574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18843bc2-49f1-4140-8075-45e8b9effa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:15.064746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4906d5e1-6a3e-4dc3-9e97-8dadf6277b20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09830ff3c16543e19a1c95791aa0dc76'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:15.065348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18843bc2-49f1-4140-8075-45e8b9effa60 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.065697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18843bc2-49f1-4140-8075-45e8b9effa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.065961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18843bc2-49f1-4140-8075-45e8b9effa60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.066597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18843bc2-49f1-4140-8075-45e8b9effa60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:15.068563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18843bc2-49f1-4140-8075-45e8b9effa60 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:15.087416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4906d5e1-6a3e-4dc3-9e97-8dadf6277b20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09830ff3c16543e19a1c95791aa0dc76'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:15.088592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4906d5e1-6a3e-4dc3-9e97-8dadf6277b20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09830ff3c16543e19a1c95791aa0dc76'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:15.104898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1088/2185] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:14 2022] DELETE /identity/v3/users/09830ff3c16543e19a1c95791aa0dc76 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:16:15.106034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1098/2186] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:15 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:16:15.109623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6f432-44e2-4317-a0d8-657964223074 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.110075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6f432-44e2-4317-a0d8-657964223074 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.110500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6f432-44e2-4317-a0d8-657964223074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.115165 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18843bc2-49f1-4140-8075-45e8b9effa60 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:15.116904 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-18843bc2-49f1-4140-8075-45e8b9effa60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:15.134468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-49e6f432-44e2-4317-a0d8-657964223074 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:15.138123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-49e6f432-44e2-4317-a0d8-657964223074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:15.201429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18843bc2-49f1-4140-8075-45e8b9effa60 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:15.208321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0781f-bb29-40d9-b803-5e333ecd3ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:15.209013 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0781f-bb29-40d9-b803-5e333ecd3ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.209301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0781f-bb29-40d9-b803-5e333ecd3ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.209591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0781f-bb29-40d9-b803-5e333ecd3ef7 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.210324 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0781f-bb29-40d9-b803-5e333ecd3ef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:15.212350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0781f-bb29-40d9-b803-5e333ecd3ef7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:15.214387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1089/2187] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:15.228684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6f432-44e2-4317-a0d8-657964223074 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:15.230503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-49e6f432-44e2-4317-a0d8-657964223074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:15.237770 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1099/2188] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:15 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:16:15.242088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0781f-bb29-40d9-b803-5e333ecd3ef7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:15.253643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0781f-bb29-40d9-b803-5e333ecd3ef7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:15.258511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e4861-d041-4399-9494-afde191b230a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:15.259211 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e4861-d041-4399-9494-afde191b230a admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.259562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e4861-d041-4399-9494-afde191b230a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.259907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e4861-d041-4399-9494-afde191b230a admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.260634 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e4861-d041-4399-9494-afde191b230a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:15.262569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e4861-d041-4399-9494-afde191b230a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:15.277878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1100/2189] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:16:15.283865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e4861-d041-4399-9494-afde191b230a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:15.295186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e4861-d041-4399-9494-afde191b230a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:15.306409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6f432-44e2-4317-a0d8-657964223074 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:15.311155 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabaaae-67b3-4798-89e3-f976a776299b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:15.311778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabaaae-67b3-4798-89e3-f976a776299b service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.312691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabaaae-67b3-4798-89e3-f976a776299b service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.312691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabaaae-67b3-4798-89e3-f976a776299b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.314851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda047e-c01d-4196-8da7-2a06978ca2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:15.315652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda047e-c01d-4196-8da7-2a06978ca2ee admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.315907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda047e-c01d-4196-8da7-2a06978ca2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.317330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda047e-c01d-4196-8da7-2a06978ca2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.319911 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda047e-c01d-4196-8da7-2a06978ca2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:15.323535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda047e-c01d-4196-8da7-2a06978ca2ee admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:15.323663 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabaaae-67b3-4798-89e3-f976a776299b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:15.325200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabaaae-67b3-4798-89e3-f976a776299b service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:15.335944 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1090/2190] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:15.340305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1101/2191] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:16:15.341412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabaaae-67b3-4798-89e3-f976a776299b service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:15.352732 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabaaae-67b3-4798-89e3-f976a776299b service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:15.358683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4eeb78-06a5-4959-b3e9-1db04726a075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:15.359540 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4eeb78-06a5-4959-b3e9-1db04726a075 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.359818 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4eeb78-06a5-4959-b3e9-1db04726a075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.360082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4eeb78-06a5-4959-b3e9-1db04726a075 admin admin] PATH_INFO: `/v3/projects/3c0901a228de4c59bc0e999f55bc1e08/users/71e0c39501bf48c2a63257a4cf75def1/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.390073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4eeb78-06a5-4959-b3e9-1db04726a075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0901a228de4c59bc0e999f55bc1e08, user_id=71e0c39501bf48c2a63257a4cf75def1, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:15.392875 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4eeb78-06a5-4959-b3e9-1db04726a075 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:15.416372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1091/2192] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:15 2022] PUT /identity/v3/projects/3c0901a228de4c59bc0e999f55bc1e08/users/71e0c39501bf48c2a63257a4cf75def1/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:15.422589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204e9e2-7abd-4026-815b-45e25bd1ad6d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.422871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204e9e2-7abd-4026-815b-45e25bd1ad6d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.423073 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204e9e2-7abd-4026-815b-45e25bd1ad6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.452404 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8204e9e2-7abd-4026-815b-45e25bd1ad6d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:15.453773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8204e9e2-7abd-4026-815b-45e25bd1ad6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:15.493219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:15.526354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1102/2193] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:15.536365 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4eeb78-06a5-4959-b3e9-1db04726a075 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:15.538436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1c4eeb78-06a5-4959-b3e9-1db04726a075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:15.574575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:15.603038 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4eeb78-06a5-4959-b3e9-1db04726a075 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:15.607592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee48cf-7619-483b-b542-2053d2aa1eac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:15.608226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee48cf-7619-483b-b542-2053d2aa1eac service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.608506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee48cf-7619-483b-b542-2053d2aa1eac service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.608812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee48cf-7619-483b-b542-2053d2aa1eac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.618947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee48cf-7619-483b-b542-2053d2aa1eac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:15.620435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee48cf-7619-483b-b542-2053d2aa1eac service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:15.631801 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1092/2194] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:15.654084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204e9e2-7abd-4026-815b-45e25bd1ad6d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:15.655918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8204e9e2-7abd-4026-815b-45e25bd1ad6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:15.683374 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204e9e2-7abd-4026-815b-45e25bd1ad6d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:15.689623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29e49a-165e-429f-8b36-c7e5a039f9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:15.690277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29e49a-165e-429f-8b36-c7e5a039f9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.690556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29e49a-165e-429f-8b36-c7e5a039f9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.690828 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29e49a-165e-429f-8b36-c7e5a039f9db admin admin] PATH_INFO: `/v3/projects/4d2fc10c7eef4dfe89030d88e3378ccc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.692219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29e49a-165e-429f-8b36-c7e5a039f9db admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d2fc10c7eef4dfe89030d88e3378ccc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:15.694086 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29e49a-165e-429f-8b36-c7e5a039f9db admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:15.770999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa29e49a-165e-429f-8b36-c7e5a039f9db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d2fc10c7eef4dfe89030d88e3378ccc', 'request_id': 'req-aa29e49a-165e-429f-8b36-c7e5a039f9db'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:15.777623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa29e49a-165e-429f-8b36-c7e5a039f9db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d2fc10c7eef4dfe89030d88e3378ccc', 'request_id': 'req-aa29e49a-165e-429f-8b36-c7e5a039f9db'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:15.784001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa29e49a-165e-429f-8b36-c7e5a039f9db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d2fc10c7eef4dfe89030d88e3378ccc', 'request_id': 'req-aa29e49a-165e-429f-8b36-c7e5a039f9db'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:15.800353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa29e49a-165e-429f-8b36-c7e5a039f9db admin admin] The token cache is being invalidate because project 4d2fc10c7eef4dfe89030d88e3378ccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:15.800679 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa29e49a-165e-429f-8b36-c7e5a039f9db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:15.801475 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aa29e49a-165e-429f-8b36-c7e5a039f9db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:15.803192 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1103/2195] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:15 2022] DELETE /identity/v3/projects/4d2fc10c7eef4dfe89030d88e3378ccc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:15.808114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f04a2-c611-469a-8b30-7eb03d2b3092 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.808398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f04a2-c611-469a-8b30-7eb03d2b3092 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.808788 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f04a2-c611-469a-8b30-7eb03d2b3092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.830355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-910f04a2-c611-469a-8b30-7eb03d2b3092 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:15.831660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-910f04a2-c611-469a-8b30-7eb03d2b3092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:15.899036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1093/2196] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:15.907619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a63bb-6381-4fd9-a5cf-1fded5276a01 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.908447 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a63bb-6381-4fd9-a5cf-1fded5276a01 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.908447 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a63bb-6381-4fd9-a5cf-1fded5276a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.932228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-300a63bb-6381-4fd9-a5cf-1fded5276a01 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:15.933563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-300a63bb-6381-4fd9-a5cf-1fded5276a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:15.942145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1104/2197] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:15.948815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f04a2-c611-469a-8b30-7eb03d2b3092 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:15.960400 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f04a2-c611-469a-8b30-7eb03d2b3092 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:15.966150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bd7b5-39c6-426f-9ead-c3ac5384a84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:15.966863 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bd7b5-39c6-426f-9ead-c3ac5384a84a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:15.967247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bd7b5-39c6-426f-9ead-c3ac5384a84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:15.967962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bd7b5-39c6-426f-9ead-c3ac5384a84a admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:15.968387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5bd7b5-39c6-426f-9ead-c3ac5384a84a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:16:15.969268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5bd7b5-39c6-426f-9ead-c3ac5384a84a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:15.970538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5bd7b5-39c6-426f-9ead-c3ac5384a84a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:15.985025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1094/2198] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:16:15.989724 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a63bb-6381-4fd9-a5cf-1fded5276a01 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:15.999551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a63bb-6381-4fd9-a5cf-1fded5276a01 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:16.004813 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dabe26-b0e3-4de0-87d1-d0a1b21b2098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:16.005585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dabe26-b0e3-4de0-87d1-d0a1b21b2098 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:16.006089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dabe26-b0e3-4de0-87d1-d0a1b21b2098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:16.006345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dabe26-b0e3-4de0-87d1-d0a1b21b2098 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:16.007027 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dabe26-b0e3-4de0-87d1-d0a1b21b2098 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:16.009390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dabe26-b0e3-4de0-87d1-d0a1b21b2098 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:16.042921 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1105/2199] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:15 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:16.047140 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bd7b5-39c6-426f-9ead-c3ac5384a84a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:16.068627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bd7b5-39c6-426f-9ead-c3ac5384a84a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:16.131289 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9795f18-d97c-4520-8ce6-02937d776849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:16.131927 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9795f18-d97c-4520-8ce6-02937d776849 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:16.132202 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9795f18-d97c-4520-8ce6-02937d776849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:16.132465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9795f18-d97c-4520-8ce6-02937d776849 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:16.133123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9795f18-d97c-4520-8ce6-02937d776849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:16.135057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9795f18-d97c-4520-8ce6-02937d776849 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:16.160723 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1095/2200] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:16 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:16.165031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dabe26-b0e3-4de0-87d1-d0a1b21b2098 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:16.177913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dabe26-b0e3-4de0-87d1-d0a1b21b2098 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:16.184191 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37271e2a-f7f1-4cba-abdf-389727843e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:16.185023 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37271e2a-f7f1-4cba-abdf-389727843e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:16.185307 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37271e2a-f7f1-4cba-abdf-389727843e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:16.185567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37271e2a-f7f1-4cba-abdf-389727843e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:16.186270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37271e2a-f7f1-4cba-abdf-389727843e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:16.187905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37271e2a-f7f1-4cba-abdf-389727843e4b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:16.202284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1106/2201] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:16.203516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9795f18-d97c-4520-8ce6-02937d776849 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:16.214248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9795f18-d97c-4520-8ce6-02937d776849 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:16.219153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c8109-0379-40a2-8503-467a982c7808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:16.219786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c8109-0379-40a2-8503-467a982c7808 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:16.220063 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c8109-0379-40a2-8503-467a982c7808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:16.220330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c8109-0379-40a2-8503-467a982c7808 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:16.221050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c8109-0379-40a2-8503-467a982c7808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:16.222632 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c8109-0379-40a2-8503-467a982c7808 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:16.233247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1096/2202] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:16.237720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37271e2a-f7f1-4cba-abdf-389727843e4b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:16.248292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37271e2a-f7f1-4cba-abdf-389727843e4b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:16.254568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a63754-6289-4a7c-a70e-fc1e6ed423c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:16.255343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a63754-6289-4a7c-a70e-fc1e6ed423c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:16.255698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a63754-6289-4a7c-a70e-fc1e6ed423c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:16.256041 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a63754-6289-4a7c-a70e-fc1e6ed423c0 admin admin] PATH_INFO: `/v3/projects/acf9183eeeef40f7a99a49d0abfeffa6/users/8dd7bb424a25489db00059483f558b9a/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:16.279509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a63754-6289-4a7c-a70e-fc1e6ed423c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf9183eeeef40f7a99a49d0abfeffa6, user_id=8dd7bb424a25489db00059483f558b9a, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:16.282313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a63754-6289-4a7c-a70e-fc1e6ed423c0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:16.296722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1107/2203] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:16 2022] PUT /identity/v3/projects/acf9183eeeef40f7a99a49d0abfeffa6/users/8dd7bb424a25489db00059483f558b9a/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:16.307301 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0e1d5-3de2-46fe-b2d6-8ba5d84c6273 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:16.307587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0e1d5-3de2-46fe-b2d6-8ba5d84c6273 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:16.307884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0e1d5-3de2-46fe-b2d6-8ba5d84c6273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:16.344452 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-64f0e1d5-3de2-46fe-b2d6-8ba5d84c6273 None None] MFA Rules not processed for user `8dd7bb424a25489db00059483f558b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:16.345836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-64f0e1d5-3de2-46fe-b2d6-8ba5d84c6273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:16.387213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:16.447552 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1097/2204] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:16.459289 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a63754-6289-4a7c-a70e-fc1e6ed423c0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:16.461313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-66a63754-6289-4a7c-a70e-fc1e6ed423c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:16.513251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21327be2-b4ce-40f3-b291-56fa3020394d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:16.514439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21327be2-b4ce-40f3-b291-56fa3020394d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:16.514915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21327be2-b4ce-40f3-b291-56fa3020394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:16.538478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a63754-6289-4a7c-a70e-fc1e6ed423c0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:16.541561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ae09f-cdb5-442a-b96c-5ddd1b7ad18e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:16.542186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ae09f-cdb5-442a-b96c-5ddd1b7ad18e service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:16.542475 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ae09f-cdb5-442a-b96c-5ddd1b7ad18e service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:16.542713 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ae09f-cdb5-442a-b96c-5ddd1b7ad18e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:16.547665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-21327be2-b4ce-40f3-b291-56fa3020394d None None] MFA Rules not processed for user `71e0c39501bf48c2a63257a4cf75def1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:16.549396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-21327be2-b4ce-40f3-b291-56fa3020394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:16.553046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ae09f-cdb5-442a-b96c-5ddd1b7ad18e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:16.555287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ae09f-cdb5-442a-b96c-5ddd1b7ad18e service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:16.559486 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:16.567785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1108/2205] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:16.627064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1098/2206] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:16.638299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ae09f-cdb5-442a-b96c-5ddd1b7ad18e service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:16.640978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-104ae09f-cdb5-442a-b96c-5ddd1b7ad18e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:16.657551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ae09f-cdb5-442a-b96c-5ddd1b7ad18e service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:16.661832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61e129-3fd8-4630-8f16-3919dbf85985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:16.662520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e129-3fd8-4630-8f16-3919dbf85985 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:16.662780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e129-3fd8-4630-8f16-3919dbf85985 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:16.663053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e129-3fd8-4630-8f16-3919dbf85985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:16.673173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61e129-3fd8-4630-8f16-3919dbf85985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:16.675339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61e129-3fd8-4630-8f16-3919dbf85985 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:16.685821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1109/2207] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:19.115030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b9231-65b0-43e3-bfe7-84eb6cc595cc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:19.115613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b9231-65b0-43e3-bfe7-84eb6cc595cc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:19.115613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b9231-65b0-43e3-bfe7-84eb6cc595cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:19.138473 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d39b9231-65b0-43e3-bfe7-84eb6cc595cc None None] MFA Rules not processed for user `8dd7bb424a25489db00059483f558b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:19.139783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d39b9231-65b0-43e3-bfe7-84eb6cc595cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:19.147752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1099/2208] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:20.221660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e894f8-6993-4bd8-a1c1-6fa2c5571cd6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:20.222335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e894f8-6993-4bd8-a1c1-6fa2c5571cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:20.222335 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e894f8-6993-4bd8-a1c1-6fa2c5571cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:20.244203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-55e894f8-6993-4bd8-a1c1-6fa2c5571cd6 None None] MFA Rules not processed for user `71e0c39501bf48c2a63257a4cf75def1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:20.245453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-55e894f8-6993-4bd8-a1c1-6fa2c5571cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:20.252674 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1110/2209] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:25.670065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b9231-65b0-43e3-bfe7-84eb6cc595cc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:25.672278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d39b9231-65b0-43e3-bfe7-84eb6cc595cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:25.745112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b9231-65b0-43e3-bfe7-84eb6cc595cc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:25.754204 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:25.754848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:25.755103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:25.755425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] PATH_INFO: `/v3/users/854cd4dd926c45519c3e32ac7e7c65a2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:25.756636 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=854cd4dd926c45519c3e32ac7e7c65a2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:25.758681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:25.809105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854cd4dd926c45519c3e32ac7e7c65a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:25.809771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854cd4dd926c45519c3e32ac7e7c65a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:25.814996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:25.817061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854cd4dd926c45519c3e32ac7e7c65a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:25.830152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854cd4dd926c45519c3e32ac7e7c65a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:25.830826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854cd4dd926c45519c3e32ac7e7c65a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:25.845530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1100/2210] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:25 2022] DELETE /identity/v3/users/854cd4dd926c45519c3e32ac7e7c65a2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:25.850276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd051ba-44f6-477c-b479-622e2689f420 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:25.850844 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd051ba-44f6-477c-b479-622e2689f420 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:25.850844 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd051ba-44f6-477c-b479-622e2689f420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:25.870950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6cd051ba-44f6-477c-b479-622e2689f420 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:25.872162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6cd051ba-44f6-477c-b479-622e2689f420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:25.943269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1111/2211] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:25.954641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:25.956837 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:26.019318 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907acdd-2a94-4d43-b67d-df886c809d40 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:26.022927 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0cda3-3ed6-46a3-87e2-b7c1dad777b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:26.023829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0cda3-3ed6-46a3-87e2-b7c1dad777b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:26.024096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0cda3-3ed6-46a3-87e2-b7c1dad777b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:26.024360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0cda3-3ed6-46a3-87e2-b7c1dad777b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:26.034665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0cda3-3ed6-46a3-87e2-b7c1dad777b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:26.036375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0cda3-3ed6-46a3-87e2-b7c1dad777b8 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:26.046716 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1101/2212] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:26.380353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd051ba-44f6-477c-b479-622e2689f420 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:26.382228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6cd051ba-44f6-477c-b479-622e2689f420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:26.400462 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd051ba-44f6-477c-b479-622e2689f420 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:26.409853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ed1db-47f8-420b-9296-643f3c865759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:26.410957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ed1db-47f8-420b-9296-643f3c865759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:26.411714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ed1db-47f8-420b-9296-643f3c865759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:26.412304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ed1db-47f8-420b-9296-643f3c865759 admin admin] PATH_INFO: `/v3/projects/e93a4e19a05f400687773aee4dbd64e3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:26.415203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ed1db-47f8-420b-9296-643f3c865759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e93a4e19a05f400687773aee4dbd64e3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:26.419337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ed1db-47f8-420b-9296-643f3c865759 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:26.480429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a45ed1db-47f8-420b-9296-643f3c865759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e93a4e19a05f400687773aee4dbd64e3', 'request_id': 'req-a45ed1db-47f8-420b-9296-643f3c865759'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:26.487200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a45ed1db-47f8-420b-9296-643f3c865759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e93a4e19a05f400687773aee4dbd64e3', 'request_id': 'req-a45ed1db-47f8-420b-9296-643f3c865759'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:26.493888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a45ed1db-47f8-420b-9296-643f3c865759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e93a4e19a05f400687773aee4dbd64e3', 'request_id': 'req-a45ed1db-47f8-420b-9296-643f3c865759'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:26.511836 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a45ed1db-47f8-420b-9296-643f3c865759 admin admin] The token cache is being invalidate because project e93a4e19a05f400687773aee4dbd64e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:26.512151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a45ed1db-47f8-420b-9296-643f3c865759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:26.513041 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a45ed1db-47f8-420b-9296-643f3c865759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:26.514714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1112/2213] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:26 2022] DELETE /identity/v3/projects/e93a4e19a05f400687773aee4dbd64e3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:26.519732 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50fd89-340b-41f9-a615-d407f3791c26 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:26.520070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50fd89-340b-41f9-a615-d407f3791c26 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:26.520352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50fd89-340b-41f9-a615-d407f3791c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:26.552815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ee50fd89-340b-41f9-a615-d407f3791c26 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:26.552815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ee50fd89-340b-41f9-a615-d407f3791c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:26.607519 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1102/2214] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:26.615257 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece8366-be11-4d23-a50a-06015cfa2e8e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:26.615540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece8366-be11-4d23-a50a-06015cfa2e8e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:26.615814 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece8366-be11-4d23-a50a-06015cfa2e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:26.636822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-eece8366-be11-4d23-a50a-06015cfa2e8e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:26.638072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-eece8366-be11-4d23-a50a-06015cfa2e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:26.646323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1113/2215] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:26.650708 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50fd89-340b-41f9-a615-d407f3791c26 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:26.660558 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50fd89-340b-41f9-a615-d407f3791c26 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:26.665120 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f7cb9-0b1c-421d-a8ac-76e27f375722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:26.665712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f7cb9-0b1c-421d-a8ac-76e27f375722 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:26.665985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f7cb9-0b1c-421d-a8ac-76e27f375722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:26.666252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f7cb9-0b1c-421d-a8ac-76e27f375722 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:26.666903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f7cb9-0b1c-421d-a8ac-76e27f375722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:16:26.667193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f7cb9-0b1c-421d-a8ac-76e27f375722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:26.668831 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f7cb9-0b1c-421d-a8ac-76e27f375722 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:26.681255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1103/2216] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:16:26.685626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece8366-be11-4d23-a50a-06015cfa2e8e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:26.695363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece8366-be11-4d23-a50a-06015cfa2e8e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:26.701329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bae12d-bc53-42f4-84ac-2ab4a5e9080d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:26.701946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bae12d-bc53-42f4-84ac-2ab4a5e9080d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:26.702323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bae12d-bc53-42f4-84ac-2ab4a5e9080d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:26.702595 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bae12d-bc53-42f4-84ac-2ab4a5e9080d admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:26.703269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bae12d-bc53-42f4-84ac-2ab4a5e9080d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:26.705248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bae12d-bc53-42f4-84ac-2ab4a5e9080d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:26.736507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1114/2217] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:26 2022] POST /identity/v3/projects => generated 357 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:26.741224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f7cb9-0b1c-421d-a8ac-76e27f375722 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:26.751636 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f7cb9-0b1c-421d-a8ac-76e27f375722 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:26.812292 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d67be2-9061-4d88-8010-0d3a9685879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:26.812959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d67be2-9061-4d88-8010-0d3a9685879b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:26.813236 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d67be2-9061-4d88-8010-0d3a9685879b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:26.813513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d67be2-9061-4d88-8010-0d3a9685879b admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:26.814196 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d67be2-9061-4d88-8010-0d3a9685879b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:26.816084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d67be2-9061-4d88-8010-0d3a9685879b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:26.826967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:26.842098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1104/2218] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:26 2022] POST /identity/v3/users => generated 279 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:26.846355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bae12d-bc53-42f4-84ac-2ab4a5e9080d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:26.857492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bae12d-bc53-42f4-84ac-2ab4a5e9080d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:26.862925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfda2c-8b52-415e-b151-6406d671b5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:26.863526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfda2c-8b52-415e-b151-6406d671b5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:26.863795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfda2c-8b52-415e-b151-6406d671b5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:26.864085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfda2c-8b52-415e-b151-6406d671b5c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:26.864766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfda2c-8b52-415e-b151-6406d671b5c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:26.866387 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfda2c-8b52-415e-b151-6406d671b5c7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:26.876921 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1115/2219] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:26.881197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d67be2-9061-4d88-8010-0d3a9685879b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:26.892797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d67be2-9061-4d88-8010-0d3a9685879b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:26.898290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cd51a-a7b2-47b6-99cf-54c31434705f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:26.899129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cd51a-a7b2-47b6-99cf-54c31434705f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:26.899545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cd51a-a7b2-47b6-99cf-54c31434705f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:26.899809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cd51a-a7b2-47b6-99cf-54c31434705f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:26.900466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cd51a-a7b2-47b6-99cf-54c31434705f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:26.902155 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cd51a-a7b2-47b6-99cf-54c31434705f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:26.915661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1105/2220] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:26.920077 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfda2c-8b52-415e-b151-6406d671b5c7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:26.930743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfda2c-8b52-415e-b151-6406d671b5c7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:26.935419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2158a-66fe-4fa3-a4c8-09336d89c64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:26.936074 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2158a-66fe-4fa3-a4c8-09336d89c64c admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:26.936416 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2158a-66fe-4fa3-a4c8-09336d89c64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:26.936740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2158a-66fe-4fa3-a4c8-09336d89c64c admin admin] PATH_INFO: `/v3/projects/8cf28095c6d14a3ea81271f34ecf9fc6/users/6c98370bc79d4502820b53f27a706550/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:26.952111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2158a-66fe-4fa3-a4c8-09336d89c64c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cf28095c6d14a3ea81271f34ecf9fc6, user_id=6c98370bc79d4502820b53f27a706550, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:26.954826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2158a-66fe-4fa3-a4c8-09336d89c64c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:26.966133 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:26.974035 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1116/2221] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:26 2022] PUT /identity/v3/projects/8cf28095c6d14a3ea81271f34ecf9fc6/users/6c98370bc79d4502820b53f27a706550/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:26.986359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2ba07-c9c7-428b-bd74-a99f3ab7c5f6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:26.986643 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2ba07-c9c7-428b-bd74-a99f3ab7c5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:26.986915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2ba07-c9c7-428b-bd74-a99f3ab7c5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:27.016210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a6c2ba07-c9c7-428b-bd74-a99f3ab7c5f6 None None] MFA Rules not processed for user `6c98370bc79d4502820b53f27a706550`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:27.017480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a6c2ba07-c9c7-428b-bd74-a99f3ab7c5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:27.094140 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1106/2222] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:27.098691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2158a-66fe-4fa3-a4c8-09336d89c64c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:27.164866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2158a-66fe-4fa3-a4c8-09336d89c64c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:27.170329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52e8e6-a56a-434b-b3db-77bd7a71ec10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:27.171132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52e8e6-a56a-434b-b3db-77bd7a71ec10 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:27.171409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52e8e6-a56a-434b-b3db-77bd7a71ec10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:27.171678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52e8e6-a56a-434b-b3db-77bd7a71ec10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:27.172341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52e8e6-a56a-434b-b3db-77bd7a71ec10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:27.174744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52e8e6-a56a-434b-b3db-77bd7a71ec10 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:27.207629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1117/2223] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:27 2022] POST /identity/v3/projects => generated 357 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:27.211946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2ba07-c9c7-428b-bd74-a99f3ab7c5f6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:27.223531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2ba07-c9c7-428b-bd74-a99f3ab7c5f6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:27.289133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20ad36-1941-480d-ac8f-fa9b4cb4dad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:27.289755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20ad36-1941-480d-ac8f-fa9b4cb4dad0 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:27.290056 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20ad36-1941-480d-ac8f-fa9b4cb4dad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:27.290328 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20ad36-1941-480d-ac8f-fa9b4cb4dad0 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:27.290997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20ad36-1941-480d-ac8f-fa9b4cb4dad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:27.293022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20ad36-1941-480d-ac8f-fa9b4cb4dad0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:27.324516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1107/2224] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:27 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:27.327305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52e8e6-a56a-434b-b3db-77bd7a71ec10 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:27.338939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52e8e6-a56a-434b-b3db-77bd7a71ec10 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:27.344709 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628e34e-ce42-4e2c-a40f-fddffdea8570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:27.345402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628e34e-ce42-4e2c-a40f-fddffdea8570 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:27.345725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628e34e-ce42-4e2c-a40f-fddffdea8570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:27.345977 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628e34e-ce42-4e2c-a40f-fddffdea8570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:27.346628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628e34e-ce42-4e2c-a40f-fddffdea8570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:27.349048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628e34e-ce42-4e2c-a40f-fddffdea8570 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:27.363262 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1118/2225] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:27.368715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20ad36-1941-480d-ac8f-fa9b4cb4dad0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:27.379734 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20ad36-1941-480d-ac8f-fa9b4cb4dad0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:27.384485 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172145e-c0ed-41e4-83bd-c6eb0d7a4683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:27.385227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172145e-c0ed-41e4-83bd-c6eb0d7a4683 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:27.385505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172145e-c0ed-41e4-83bd-c6eb0d7a4683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:27.385773 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172145e-c0ed-41e4-83bd-c6eb0d7a4683 admin admin] PATH_INFO: `/v3/projects/a44d448fb88449d9af983a507b442231/users/d730d262a154437fbd9f22f702765a6d/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:27.401807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172145e-c0ed-41e4-83bd-c6eb0d7a4683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44d448fb88449d9af983a507b442231, user_id=d730d262a154437fbd9f22f702765a6d, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:27.404468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172145e-c0ed-41e4-83bd-c6eb0d7a4683 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:27.418621 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1108/2226] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:27 2022] PUT /identity/v3/projects/a44d448fb88449d9af983a507b442231/users/d730d262a154437fbd9f22f702765a6d/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:27.425030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628e34e-ce42-4e2c-a40f-fddffdea8570 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:27.437539 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628e34e-ce42-4e2c-a40f-fddffdea8570 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:27.507336 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69ac88-bf4c-4502-a41a-83beca12de4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:27.507957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69ac88-bf4c-4502-a41a-83beca12de4f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:27.508234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69ac88-bf4c-4502-a41a-83beca12de4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:27.509248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69ac88-bf4c-4502-a41a-83beca12de4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:27.509901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69ac88-bf4c-4502-a41a-83beca12de4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:27.511529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69ac88-bf4c-4502-a41a-83beca12de4f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:27.523273 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1119/2227] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:27.527475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172145e-c0ed-41e4-83bd-c6eb0d7a4683 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:27.538654 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172145e-c0ed-41e4-83bd-c6eb0d7a4683 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:27.544103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbda21-00b6-4b9e-8bf0-b947c57488bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:27.544864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbda21-00b6-4b9e-8bf0-b947c57488bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:27.545120 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbda21-00b6-4b9e-8bf0-b947c57488bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:27.545395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbda21-00b6-4b9e-8bf0-b947c57488bb admin admin] PATH_INFO: `/v3/projects/a44d448fb88449d9af983a507b442231/users/d730d262a154437fbd9f22f702765a6d/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:27.547575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbda21-00b6-4b9e-8bf0-b947c57488bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44d448fb88449d9af983a507b442231, user_id=d730d262a154437fbd9f22f702765a6d, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:27.550613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbda21-00b6-4b9e-8bf0-b947c57488bb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:27.571294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1109/2228] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:27 2022] PUT /identity/v3/projects/a44d448fb88449d9af983a507b442231/users/d730d262a154437fbd9f22f702765a6d/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:27.572939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cc25e-28de-4419-b1f1-ca51c4b4a7b0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:27.573225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cc25e-28de-4419-b1f1-ca51c4b4a7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:27.573571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cc25e-28de-4419-b1f1-ca51c4b4a7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:27.610412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ca3cc25e-28de-4419-b1f1-ca51c4b4a7b0 None None] MFA Rules not processed for user `d730d262a154437fbd9f22f702765a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:27.611678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ca3cc25e-28de-4419-b1f1-ca51c4b4a7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:27.682339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:27.693178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1120/2229] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:27.705246 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbda21-00b6-4b9e-8bf0-b947c57488bb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:27.707016 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-80cbda21-00b6-4b9e-8bf0-b947c57488bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:27.733192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:27.765457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbda21-00b6-4b9e-8bf0-b947c57488bb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:27.769409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e7ea8-7d7e-4279-b464-b2ffd01a9a40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:27.770004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e7ea8-7d7e-4279-b464-b2ffd01a9a40 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:27.770288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e7ea8-7d7e-4279-b464-b2ffd01a9a40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:27.770556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e7ea8-7d7e-4279-b464-b2ffd01a9a40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:27.779802 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e7ea8-7d7e-4279-b464-b2ffd01a9a40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:27.781336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e7ea8-7d7e-4279-b464-b2ffd01a9a40 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:27.793515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1110/2230] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:27 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:27.876491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cc25e-28de-4419-b1f1-ca51c4b4a7b0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:27.886940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cc25e-28de-4419-b1f1-ca51c4b4a7b0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:27.890758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1d29d-0b32-4050-a817-9b248bad4477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:27.891375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1d29d-0b32-4050-a817-9b248bad4477 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:27.891649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1d29d-0b32-4050-a817-9b248bad4477 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:27.891917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1d29d-0b32-4050-a817-9b248bad4477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:27.958029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1d29d-0b32-4050-a817-9b248bad4477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:27.959485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1d29d-0b32-4050-a817-9b248bad4477 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:27.968930 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1121/2231] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:37.786515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=16 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:37.964416 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=31 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:43.369487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e7ea8-7d7e-4279-b464-b2ffd01a9a40 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:43.371100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-db6e7ea8-7d7e-4279-b464-b2ffd01a9a40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:43.468695 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e7ea8-7d7e-4279-b464-b2ffd01a9a40 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:43.473363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:43.474035 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:43.474308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:43.474579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] PATH_INFO: `/v3/users/7b3ed1be79a84ee1bb1307d70ddf59bc` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:43.475732 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b3ed1be79a84ee1bb1307d70ddf59bc)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:43.477762 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:43.535736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b3ed1be79a84ee1bb1307d70ddf59bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:43.536577 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b3ed1be79a84ee1bb1307d70ddf59bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:43.546896 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b3ed1be79a84ee1bb1307d70ddf59bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:43.559696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b3ed1be79a84ee1bb1307d70ddf59bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:43.560626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b3ed1be79a84ee1bb1307d70ddf59bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:43.581428 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1111/2232] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:43 2022] DELETE /identity/v3/users/7b3ed1be79a84ee1bb1307d70ddf59bc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:16:43.584123 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd01d9a-bc9b-43a5-9cf2-e2dd62053b50 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:43.584639 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd01d9a-bc9b-43a5-9cf2-e2dd62053b50 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:43.584755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd01d9a-bc9b-43a5-9cf2-e2dd62053b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:43.609432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1cd01d9a-bc9b-43a5-9cf2-e2dd62053b50 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:43.610835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1cd01d9a-bc9b-43a5-9cf2-e2dd62053b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:43.689443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1122/2233] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:43.700501 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:43.702544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:43.796202 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8b199-bd9c-4f9c-8937-61f59e93efbc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:43.802005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3af34b-6fac-403c-b44a-fcc0bfd9e704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:43.802769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3af34b-6fac-403c-b44a-fcc0bfd9e704 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:43.803112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3af34b-6fac-403c-b44a-fcc0bfd9e704 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:43.803554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3af34b-6fac-403c-b44a-fcc0bfd9e704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:43.814935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3af34b-6fac-403c-b44a-fcc0bfd9e704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:43.816717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3af34b-6fac-403c-b44a-fcc0bfd9e704 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:43.828409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1112/2234] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:44.179995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd01d9a-bc9b-43a5-9cf2-e2dd62053b50 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:44.182218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1cd01d9a-bc9b-43a5-9cf2-e2dd62053b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:44.201140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd01d9a-bc9b-43a5-9cf2-e2dd62053b50 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:44.211892 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc67858-3295-4cda-8a92-82aff70b171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:44.213085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc67858-3295-4cda-8a92-82aff70b171e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:44.213718 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc67858-3295-4cda-8a92-82aff70b171e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:44.215007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc67858-3295-4cda-8a92-82aff70b171e admin admin] PATH_INFO: `/v3/projects/526fbf7a6c6d4d42a1b1353e80710c42` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:44.216906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc67858-3295-4cda-8a92-82aff70b171e admin admin] RBAC: Authorizing `identity:delete_project(project_id=526fbf7a6c6d4d42a1b1353e80710c42)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:44.219278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc67858-3295-4cda-8a92-82aff70b171e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:44.319430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7cc67858-3295-4cda-8a92-82aff70b171e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '526fbf7a6c6d4d42a1b1353e80710c42', 'request_id': 'req-7cc67858-3295-4cda-8a92-82aff70b171e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:44.322722 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7cc67858-3295-4cda-8a92-82aff70b171e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '526fbf7a6c6d4d42a1b1353e80710c42', 'request_id': 'req-7cc67858-3295-4cda-8a92-82aff70b171e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:44.334520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7cc67858-3295-4cda-8a92-82aff70b171e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '526fbf7a6c6d4d42a1b1353e80710c42', 'request_id': 'req-7cc67858-3295-4cda-8a92-82aff70b171e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:44.360836 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7cc67858-3295-4cda-8a92-82aff70b171e admin admin] The token cache is being invalidate because project 526fbf7a6c6d4d42a1b1353e80710c42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:44.361383 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7cc67858-3295-4cda-8a92-82aff70b171e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:44.362391 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7cc67858-3295-4cda-8a92-82aff70b171e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:44.366500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1123/2235] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:44 2022] DELETE /identity/v3/projects/526fbf7a6c6d4d42a1b1353e80710c42 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:44.373199 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e8d35-5972-4620-8a1b-6dda68e5e5df None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:44.374406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e8d35-5972-4620-8a1b-6dda68e5e5df None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:44.375475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e8d35-5972-4620-8a1b-6dda68e5e5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:44.405661 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7b9e8d35-5972-4620-8a1b-6dda68e5e5df None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:44.406746 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7b9e8d35-5972-4620-8a1b-6dda68e5e5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:44.428884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:44.494037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1113/2236] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:44.502252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139b6b2-80d8-4f10-9776-c97db8ac3cda None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:44.502574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139b6b2-80d8-4f10-9776-c97db8ac3cda None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:44.502831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139b6b2-80d8-4f10-9776-c97db8ac3cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:44.524807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4139b6b2-80d8-4f10-9776-c97db8ac3cda None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:44.526044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4139b6b2-80d8-4f10-9776-c97db8ac3cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:44.535251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1124/2237] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:44.539867 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e8d35-5972-4620-8a1b-6dda68e5e5df None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:44.550669 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e8d35-5972-4620-8a1b-6dda68e5e5df None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:44.555426 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e13f2-4733-4035-ac45-7aa92d1a41a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:44.556034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e13f2-4733-4035-ac45-7aa92d1a41a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:44.556379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e13f2-4733-4035-ac45-7aa92d1a41a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:44.556614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e13f2-4733-4035-ac45-7aa92d1a41a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:44.557298 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e13f2-4733-4035-ac45-7aa92d1a41a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:16:44.557603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e13f2-4733-4035-ac45-7aa92d1a41a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:44.559207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e13f2-4733-4035-ac45-7aa92d1a41a5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:44.570866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1114/2238] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:16:44.575528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139b6b2-80d8-4f10-9776-c97db8ac3cda None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:44.585524 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139b6b2-80d8-4f10-9776-c97db8ac3cda None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:44.590976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd568ab-8629-439f-82c3-ed957764f221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:44.591574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd568ab-8629-439f-82c3-ed957764f221 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:44.591863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd568ab-8629-439f-82c3-ed957764f221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:44.592133 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd568ab-8629-439f-82c3-ed957764f221 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:44.592815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd568ab-8629-439f-82c3-ed957764f221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:44.594826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd568ab-8629-439f-82c3-ed957764f221 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:44.625279 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1125/2239] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:44 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:44.629949 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e13f2-4733-4035-ac45-7aa92d1a41a5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:44.640722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e13f2-4733-4035-ac45-7aa92d1a41a5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:44.707151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b3bbb-a2b4-4b9f-bfe9-b28ae197504d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:44.707776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b3bbb-a2b4-4b9f-bfe9-b28ae197504d admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:44.708093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b3bbb-a2b4-4b9f-bfe9-b28ae197504d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:44.708340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b3bbb-a2b4-4b9f-bfe9-b28ae197504d admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:44.709064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b3bbb-a2b4-4b9f-bfe9-b28ae197504d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:44.711055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b3bbb-a2b4-4b9f-bfe9-b28ae197504d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:44.738332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1115/2240] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:44 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:44.739530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd568ab-8629-439f-82c3-ed957764f221 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:44.750124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd568ab-8629-439f-82c3-ed957764f221 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:44.762202 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac779de-98ba-47cc-af9a-f6ef2a337aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:44.763128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac779de-98ba-47cc-af9a-f6ef2a337aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:44.763809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac779de-98ba-47cc-af9a-f6ef2a337aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:44.764352 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac779de-98ba-47cc-af9a-f6ef2a337aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:44.765227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac779de-98ba-47cc-af9a-f6ef2a337aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:44.766865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac779de-98ba-47cc-af9a-f6ef2a337aa3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:44.779561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1126/2241] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:44.784538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b3bbb-a2b4-4b9f-bfe9-b28ae197504d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:44.799071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b3bbb-a2b4-4b9f-bfe9-b28ae197504d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:44.805916 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844e7f4-685c-4e1e-a057-3cafb4f4ab90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:44.806586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844e7f4-685c-4e1e-a057-3cafb4f4ab90 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:44.806804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844e7f4-685c-4e1e-a057-3cafb4f4ab90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:44.807068 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844e7f4-685c-4e1e-a057-3cafb4f4ab90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:44.807962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844e7f4-685c-4e1e-a057-3cafb4f4ab90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:44.809682 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844e7f4-685c-4e1e-a057-3cafb4f4ab90 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:44.827364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1116/2242] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:44.831792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac779de-98ba-47cc-af9a-f6ef2a337aa3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:44.844224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac779de-98ba-47cc-af9a-f6ef2a337aa3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:44.849322 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2bf4c-3bd2-4f87-9082-86a4bc37f68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:44.850009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2bf4c-3bd2-4f87-9082-86a4bc37f68b admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:44.850310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2bf4c-3bd2-4f87-9082-86a4bc37f68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:44.850577 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2bf4c-3bd2-4f87-9082-86a4bc37f68b admin admin] PATH_INFO: `/v3/projects/70e949fdb7c14e6993449522f16ba84e/users/17aa0d27da234aeb957e1ef96fe61413/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:44.866645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2bf4c-3bd2-4f87-9082-86a4bc37f68b admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e949fdb7c14e6993449522f16ba84e, user_id=17aa0d27da234aeb957e1ef96fe61413, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:44.869450 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2bf4c-3bd2-4f87-9082-86a4bc37f68b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:44.884128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1127/2243] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:44 2022] PUT /identity/v3/projects/70e949fdb7c14e6993449522f16ba84e/users/17aa0d27da234aeb957e1ef96fe61413/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:44.894691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fa224-2de5-4ccc-898a-0981a6020a61 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:44.894976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fa224-2de5-4ccc-898a-0981a6020a61 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:44.895248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fa224-2de5-4ccc-898a-0981a6020a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:44.929365 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-955fa224-2de5-4ccc-898a-0981a6020a61 None None] MFA Rules not processed for user `17aa0d27da234aeb957e1ef96fe61413`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:44.930960 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-955fa224-2de5-4ccc-898a-0981a6020a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:45.016637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1117/2244] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:45.027915 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2bf4c-3bd2-4f87-9082-86a4bc37f68b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:45.029989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-36e2bf4c-3bd2-4f87-9082-86a4bc37f68b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:45.056055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:45.103355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2bf4c-3bd2-4f87-9082-86a4bc37f68b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:45.107845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692c890-7376-4403-ac78-ee58fb9361a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:45.108480 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692c890-7376-4403-ac78-ee58fb9361a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:45.108789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692c890-7376-4403-ac78-ee58fb9361a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:45.109064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692c890-7376-4403-ac78-ee58fb9361a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:45.120181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692c890-7376-4403-ac78-ee58fb9361a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:45.125130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692c890-7376-4403-ac78-ee58fb9361a6 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:45.133386 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1128/2245] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:52.551740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78fa2-e37d-4d4b-87c5-c0a5c52f28e0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:52.552193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78fa2-e37d-4d4b-87c5-c0a5c52f28e0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:52.552438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78fa2-e37d-4d4b-87c5-c0a5c52f28e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:52.573141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6ff78fa2-e37d-4d4b-87c5-c0a5c52f28e0 None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:52.574472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6ff78fa2-e37d-4d4b-87c5-c0a5c52f28e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:52.595022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:52.637204 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1118/2246] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:16:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:52.697806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692c890-7376-4403-ac78-ee58fb9361a6 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:52.700776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1692c890-7376-4403-ac78-ee58fb9361a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:52.867510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692c890-7376-4403-ac78-ee58fb9361a6 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:52.907061 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56a871-ce6f-4773-8966-70351cbd02b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:52.907935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56a871-ce6f-4773-8966-70351cbd02b6 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:52.908286 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56a871-ce6f-4773-8966-70351cbd02b6 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:52.908625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56a871-ce6f-4773-8966-70351cbd02b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:52.929983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56a871-ce6f-4773-8966-70351cbd02b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:52.931667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56a871-ce6f-4773-8966-70351cbd02b6 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:52.957133 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1129/2247] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:16:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:16:53.313750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff78fa2-e37d-4d4b-87c5-c0a5c52f28e0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:53.315557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6ff78fa2-e37d-4d4b-87c5-c0a5c52f28e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:53.425066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff78fa2-e37d-4d4b-87c5-c0a5c52f28e0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:53.430758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:53.431488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:53.431845 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:53.432309 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] PATH_INFO: `/v3/users/6c98370bc79d4502820b53f27a706550` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:53.433821 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c98370bc79d4502820b53f27a706550)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:53.435794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:53.497694 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c98370bc79d4502820b53f27a706550'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:53.498501 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c98370bc79d4502820b53f27a706550'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:53.506537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c98370bc79d4502820b53f27a706550'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:53.520071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c98370bc79d4502820b53f27a706550'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:53.520938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c98370bc79d4502820b53f27a706550'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:53.535792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1119/2248] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:53 2022] DELETE /identity/v3/users/6c98370bc79d4502820b53f27a706550 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:53.540305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138601d-f92b-462b-b03b-b6f34ddee0c7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:53.540596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138601d-f92b-462b-b03b-b6f34ddee0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:53.540956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138601d-f92b-462b-b03b-b6f34ddee0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:53.560736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3138601d-f92b-462b-b03b-b6f34ddee0c7 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:53.561961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3138601d-f92b-462b-b03b-b6f34ddee0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:53.630765 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1130/2249] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:53.643610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:53.645184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:53.698382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:53.712515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa70f09-10ed-466f-b16c-c62afb7fe314 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:53.716621 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819cce9-d401-4356-87aa-4fcd9278cec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:53.717279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819cce9-d401-4356-87aa-4fcd9278cec0 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:53.717637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819cce9-d401-4356-87aa-4fcd9278cec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:53.717902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819cce9-d401-4356-87aa-4fcd9278cec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:53.727490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defafd5b-3941-40dc-b700-5e9f86af104e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:53.727991 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defafd5b-3941-40dc-b700-5e9f86af104e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:53.728352 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defafd5b-3941-40dc-b700-5e9f86af104e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:53.735259 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819cce9-d401-4356-87aa-4fcd9278cec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:53.737046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819cce9-d401-4356-87aa-4fcd9278cec0 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:53.750113 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1120/2250] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:16:53.761850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-defafd5b-3941-40dc-b700-5e9f86af104e None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:53.764287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-defafd5b-3941-40dc-b700-5e9f86af104e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:53.823194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:53.842860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1131/2251] 10.222.0.79 () {58 vars in 967 bytes} [Tue Sep 6 21:16:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:53.860492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819cce9-d401-4356-87aa-4fcd9278cec0 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:53.861931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3819cce9-d401-4356-87aa-4fcd9278cec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:53.981151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819cce9-d401-4356-87aa-4fcd9278cec0 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:53.986991 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a274a-215f-47a5-ba84-3f6d1c1f1710 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:53.987609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a274a-215f-47a5-ba84-3f6d1c1f1710 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:53.988134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a274a-215f-47a5-ba84-3f6d1c1f1710 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:53.988414 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a274a-215f-47a5-ba84-3f6d1c1f1710 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:54.003387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a274a-215f-47a5-ba84-3f6d1c1f1710 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:54.005918 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a274a-215f-47a5-ba84-3f6d1c1f1710 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:54.018991 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1121/2252] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:16:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:16:54.101028 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defafd5b-3941-40dc-b700-5e9f86af104e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:54.103483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-defafd5b-3941-40dc-b700-5e9f86af104e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:54.127242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defafd5b-3941-40dc-b700-5e9f86af104e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:54.139117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673554be-6fbf-4501-a0d9-cb386464845e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:54.140114 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673554be-6fbf-4501-a0d9-cb386464845e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:54.140855 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673554be-6fbf-4501-a0d9-cb386464845e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:54.140956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673554be-6fbf-4501-a0d9-cb386464845e admin admin] PATH_INFO: `/v3/projects/8cf28095c6d14a3ea81271f34ecf9fc6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:54.142366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673554be-6fbf-4501-a0d9-cb386464845e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cf28095c6d14a3ea81271f34ecf9fc6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:54.146347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673554be-6fbf-4501-a0d9-cb386464845e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:54.255113 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-673554be-6fbf-4501-a0d9-cb386464845e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cf28095c6d14a3ea81271f34ecf9fc6', 'request_id': 'req-673554be-6fbf-4501-a0d9-cb386464845e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:54.262067 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-673554be-6fbf-4501-a0d9-cb386464845e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cf28095c6d14a3ea81271f34ecf9fc6', 'request_id': 'req-673554be-6fbf-4501-a0d9-cb386464845e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:54.268033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-673554be-6fbf-4501-a0d9-cb386464845e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cf28095c6d14a3ea81271f34ecf9fc6', 'request_id': 'req-673554be-6fbf-4501-a0d9-cb386464845e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:54.281697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-673554be-6fbf-4501-a0d9-cb386464845e admin admin] The token cache is being invalidate because project 8cf28095c6d14a3ea81271f34ecf9fc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:54.282254 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-673554be-6fbf-4501-a0d9-cb386464845e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:54.282882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-673554be-6fbf-4501-a0d9-cb386464845e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:54.284460 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1132/2253] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:54 2022] DELETE /identity/v3/projects/8cf28095c6d14a3ea81271f34ecf9fc6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:54.289029 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a274a-215f-47a5-ba84-3f6d1c1f1710 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:54.308491 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a274a-215f-47a5-ba84-3f6d1c1f1710 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:54.372085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:54.372758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:54.373054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:54.373325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] PATH_INFO: `/v3/users/d730d262a154437fbd9f22f702765a6d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:54.379529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d730d262a154437fbd9f22f702765a6d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:54.381623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:54.384935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074da348-3d4c-41e7-9fd7-bb08f3d81d75 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:54.385630 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074da348-3d4c-41e7-9fd7-bb08f3d81d75 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:54.385906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074da348-3d4c-41e7-9fd7-bb08f3d81d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:54.412566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-074da348-3d4c-41e7-9fd7-bb08f3d81d75 None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:54.413933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-074da348-3d4c-41e7-9fd7-bb08f3d81d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:54.444562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd730d262a154437fbd9f22f702765a6d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:54.446392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd730d262a154437fbd9f22f702765a6d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:54.453826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd730d262a154437fbd9f22f702765a6d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:54.470436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd730d262a154437fbd9f22f702765a6d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:54.471816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd730d262a154437fbd9f22f702765a6d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:54.475883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:54.494631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1133/2254] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:54.498950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1122/2255] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:54 2022] DELETE /identity/v3/users/d730d262a154437fbd9f22f702765a6d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:54.508336 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074da348-3d4c-41e7-9fd7-bb08f3d81d75 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:54.510253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-074da348-3d4c-41e7-9fd7-bb08f3d81d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:54.592058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074da348-3d4c-41e7-9fd7-bb08f3d81d75 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:54.620287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:54.664025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa6e89-04f2-4940-97c2-f3da942f59a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:54.664648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa6e89-04f2-4940-97c2-f3da942f59a1 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:54.664949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa6e89-04f2-4940-97c2-f3da942f59a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:54.665209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa6e89-04f2-4940-97c2-f3da942f59a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:54.730690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa6e89-04f2-4940-97c2-f3da942f59a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:54.732333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa6e89-04f2-4940-97c2-f3da942f59a1 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:54.740920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1134/2256] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:16:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:16:54.893990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:54.897933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:54.993366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8833da8-1f3c-4c0b-aafa-1331dbb8569f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:54.999008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d4982-e6aa-4daf-b795-0ace238b2e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:54.999909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d4982-e6aa-4daf-b795-0ace238b2e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.000233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d4982-e6aa-4daf-b795-0ace238b2e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.005168 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d4982-e6aa-4daf-b795-0ace238b2e00 admin admin] PATH_INFO: `/v3/projects/a44d448fb88449d9af983a507b442231` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.006808 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d4982-e6aa-4daf-b795-0ace238b2e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a44d448fb88449d9af983a507b442231)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:55.010883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d4982-e6aa-4daf-b795-0ace238b2e00 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:55.081061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1b8d4982-e6aa-4daf-b795-0ace238b2e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a44d448fb88449d9af983a507b442231', 'request_id': 'req-1b8d4982-e6aa-4daf-b795-0ace238b2e00'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:55.100396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1b8d4982-e6aa-4daf-b795-0ace238b2e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a44d448fb88449d9af983a507b442231', 'request_id': 'req-1b8d4982-e6aa-4daf-b795-0ace238b2e00'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:55.108225 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1b8d4982-e6aa-4daf-b795-0ace238b2e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a44d448fb88449d9af983a507b442231', 'request_id': 'req-1b8d4982-e6aa-4daf-b795-0ace238b2e00'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:55.129669 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1b8d4982-e6aa-4daf-b795-0ace238b2e00 admin admin] The token cache is being invalidate because project a44d448fb88449d9af983a507b442231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:55.129669 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1b8d4982-e6aa-4daf-b795-0ace238b2e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:55.129669 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1b8d4982-e6aa-4daf-b795-0ace238b2e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:55.129669 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1123/2257] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:54 2022] DELETE /identity/v3/projects/a44d448fb88449d9af983a507b442231 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:16:55.134100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24cbb9-5e3e-457b-9479-534d24a34116 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.134459 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24cbb9-5e3e-457b-9479-534d24a34116 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.135150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24cbb9-5e3e-457b-9479-534d24a34116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.147757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5a27-349d-40d9-b314-3c2739067118 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.148298 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5a27-349d-40d9-b314-3c2739067118 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.148854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5a27-349d-40d9-b314-3c2739067118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.163726 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4f24cbb9-5e3e-457b-9479-534d24a34116 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:55.165247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4f24cbb9-5e3e-457b-9479-534d24a34116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:55.175858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d1da5a27-349d-40d9-b314-3c2739067118 None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:55.177238 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d1da5a27-349d-40d9-b314-3c2739067118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:55.245669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1135/2258] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:55.254052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81201bdc-5dc8-4aef-ae54-e15dbfbb2e17 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.254710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81201bdc-5dc8-4aef-ae54-e15dbfbb2e17 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.255237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81201bdc-5dc8-4aef-ae54-e15dbfbb2e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.268434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1124/2259] 10.222.0.79 () {58 vars in 967 bytes} [Tue Sep 6 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:55.276587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da5a27-349d-40d9-b314-3c2739067118 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:55.278307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d1da5a27-349d-40d9-b314-3c2739067118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:55.303867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-81201bdc-5dc8-4aef-ae54-e15dbfbb2e17 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:55.305210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-81201bdc-5dc8-4aef-ae54-e15dbfbb2e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:55.315310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1136/2260] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:55.318626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:55.319480 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81201bdc-5dc8-4aef-ae54-e15dbfbb2e17 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:55.329931 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81201bdc-5dc8-4aef-ae54-e15dbfbb2e17 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:55.335114 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523793a7-e1a6-431d-8abd-cee87457e5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:55.335735 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523793a7-e1a6-431d-8abd-cee87457e5da admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.336016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523793a7-e1a6-431d-8abd-cee87457e5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.336294 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523793a7-e1a6-431d-8abd-cee87457e5da admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.336990 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523793a7-e1a6-431d-8abd-cee87457e5da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:16:55.337294 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523793a7-e1a6-431d-8abd-cee87457e5da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:55.339013 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523793a7-e1a6-431d-8abd-cee87457e5da admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:55.352602 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1137/2261] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:16:55.357184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523793a7-e1a6-431d-8abd-cee87457e5da admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:55.361695 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da5a27-349d-40d9-b314-3c2739067118 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:55.367584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e52868-6392-4164-ae07-207984d595fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:55.368158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e52868-6392-4164-ae07-207984d595fa service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.368432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e52868-6392-4164-ae07-207984d595fa service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.368743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e52868-6392-4164-ae07-207984d595fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.373688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523793a7-e1a6-431d-8abd-cee87457e5da admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:55.378816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e52868-6392-4164-ae07-207984d595fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:55.379234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f1d32-40b6-43b8-9acb-660b9e41cc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:55.379973 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f1d32-40b6-43b8-9acb-660b9e41cc03 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.380194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f1d32-40b6-43b8-9acb-660b9e41cc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.380450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e52868-6392-4164-ae07-207984d595fa service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:55.380570 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f1d32-40b6-43b8-9acb-660b9e41cc03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.381125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f1d32-40b6-43b8-9acb-660b9e41cc03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:55.383034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f1d32-40b6-43b8-9acb-660b9e41cc03 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:55.390364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1125/2262] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:16:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:16:55.427145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1138/2263] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:55 2022] POST /identity/v3/projects => generated 407 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:55.431487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e52868-6392-4164-ae07-207984d595fa service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:55.443554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e52868-6392-4164-ae07-207984d595fa service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:55.523176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6f0c9-0120-4779-baf5-305671c4b8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:55.523949 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6f0c9-0120-4779-baf5-305671c4b8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.524237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6f0c9-0120-4779-baf5-305671c4b8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.524628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6f0c9-0120-4779-baf5-305671c4b8c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.525501 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6f0c9-0120-4779-baf5-305671c4b8c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:55.528018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6f0c9-0120-4779-baf5-305671c4b8c4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:55.540375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f1d32-40b6-43b8-9acb-660b9e41cc03 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:55.543480 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a38f1d32-40b6-43b8-9acb-660b9e41cc03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:55.564089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f1d32-40b6-43b8-9acb-660b9e41cc03 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:55.564911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1126/2264] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:55 2022] POST /identity/v3/users => generated 304 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:55.569156 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6f0c9-0120-4779-baf5-305671c4b8c4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:55.580353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb0764-9300-473f-972d-a1fa5ba11fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:55.581825 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb0764-9300-473f-972d-a1fa5ba11fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.582268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb0764-9300-473f-972d-a1fa5ba11fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.582747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb0764-9300-473f-972d-a1fa5ba11fd7 admin admin] PATH_INFO: `/v3/users/20229315384e45ca83b397307c593708` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.584098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb0764-9300-473f-972d-a1fa5ba11fd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20229315384e45ca83b397307c593708)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:55.586480 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb0764-9300-473f-972d-a1fa5ba11fd7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:55.589962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6f0c9-0120-4779-baf5-305671c4b8c4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:55.615727 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fc7d8-2461-4326-930d-9456ea255c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:55.616500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fc7d8-2461-4326-930d-9456ea255c4f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.616810 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fc7d8-2461-4326-930d-9456ea255c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.617083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fc7d8-2461-4326-930d-9456ea255c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.617852 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fc7d8-2461-4326-930d-9456ea255c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:55.621755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fc7d8-2461-4326-930d-9456ea255c4f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:55.643640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1127/2265] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:55.653372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fc7d8-2461-4326-930d-9456ea255c4f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:55.653701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:55.666440 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fc7d8-2461-4326-930d-9456ea255c4f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:55.672744 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6e111-153f-4353-a6eb-36e9ba8c0ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:55.673403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6e111-153f-4353-a6eb-36e9ba8c0ece admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.674848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6e111-153f-4353-a6eb-36e9ba8c0ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.675126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6e111-153f-4353-a6eb-36e9ba8c0ece admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.675786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6e111-153f-4353-a6eb-36e9ba8c0ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:55.680463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6e111-153f-4353-a6eb-36e9ba8c0ece admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:55.696899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1128/2266] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:55.701012 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6e111-153f-4353-a6eb-36e9ba8c0ece admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:55.704615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fbbb0764-9300-473f-972d-a1fa5ba11fd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20229315384e45ca83b397307c593708'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:55.705287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fbbb0764-9300-473f-972d-a1fa5ba11fd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20229315384e45ca83b397307c593708'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:55.716246 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6e111-153f-4353-a6eb-36e9ba8c0ece admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:55.720974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fbbb0764-9300-473f-972d-a1fa5ba11fd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20229315384e45ca83b397307c593708'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:55.721989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be64833-faf4-45de-b43b-d8878344e868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:55.722745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be64833-faf4-45de-b43b-d8878344e868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.723091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be64833-faf4-45de-b43b-d8878344e868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.723465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be64833-faf4-45de-b43b-d8878344e868 admin admin] PATH_INFO: `/v3/projects/761ca42525ff4a8f8b8bcaf6ff364320/users/cc4824e9629d4f1caf04a7270ea73ba0/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.746055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be64833-faf4-45de-b43b-d8878344e868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=761ca42525ff4a8f8b8bcaf6ff364320, user_id=cc4824e9629d4f1caf04a7270ea73ba0, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:55.750935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be64833-faf4-45de-b43b-d8878344e868 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:55.757824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fbbb0764-9300-473f-972d-a1fa5ba11fd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20229315384e45ca83b397307c593708'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:55.760598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fbbb0764-9300-473f-972d-a1fa5ba11fd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20229315384e45ca83b397307c593708'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:55.776031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1129/2267] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:55 2022] PUT /identity/v3/projects/761ca42525ff4a8f8b8bcaf6ff364320/users/cc4824e9629d4f1caf04a7270ea73ba0/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:55.798987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7926d-122b-4b14-a1d9-a6d6cb3fca16 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.799375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7926d-122b-4b14-a1d9-a6d6cb3fca16 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.799812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7926d-122b-4b14-a1d9-a6d6cb3fca16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.806305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1139/2268] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:55 2022] DELETE /identity/v3/users/20229315384e45ca83b397307c593708 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:55.807956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204bbbf-d435-4e92-86df-f44c579cc8c0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:55.808239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204bbbf-d435-4e92-86df-f44c579cc8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:55.808509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204bbbf-d435-4e92-86df-f44c579cc8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:55.838769 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1204bbbf-d435-4e92-86df-f44c579cc8c0 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:55.840155 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1204bbbf-d435-4e92-86df-f44c579cc8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:55.871491 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fec7926d-122b-4b14-a1d9-a6d6cb3fca16 None None] MFA Rules not processed for user `cc4824e9629d4f1caf04a7270ea73ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:55.872995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fec7926d-122b-4b14-a1d9-a6d6cb3fca16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:55.927014 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1140/2269] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:55.937818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204bbbf-d435-4e92-86df-f44c579cc8c0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:55.939310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1204bbbf-d435-4e92-86df-f44c579cc8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:55.965315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:55.992705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1130/2270] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:55.997367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7926d-122b-4b14-a1d9-a6d6cb3fca16 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:56.013671 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fec7926d-122b-4b14-a1d9-a6d6cb3fca16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:56.032600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204bbbf-d435-4e92-86df-f44c579cc8c0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:56.036415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc611a51-5ac7-42f4-bf39-a2c3812ee2e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:56.037049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc611a51-5ac7-42f4-bf39-a2c3812ee2e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.037337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc611a51-5ac7-42f4-bf39-a2c3812ee2e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.037588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc611a51-5ac7-42f4-bf39-a2c3812ee2e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.044621 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7926d-122b-4b14-a1d9-a6d6cb3fca16 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:56.049777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc611a51-5ac7-42f4-bf39-a2c3812ee2e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:56.060716 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc611a51-5ac7-42f4-bf39-a2c3812ee2e9 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:56.060955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589b4ac-ea71-4a6e-9e7d-33e476dd1177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:56.060955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589b4ac-ea71-4a6e-9e7d-33e476dd1177 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.060955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589b4ac-ea71-4a6e-9e7d-33e476dd1177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.060955 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589b4ac-ea71-4a6e-9e7d-33e476dd1177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.061337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589b4ac-ea71-4a6e-9e7d-33e476dd1177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:56.061337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589b4ac-ea71-4a6e-9e7d-33e476dd1177 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:56.062507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1141/2271] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:16:56.107734 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1131/2272] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:55 2022] POST /identity/v3/projects => generated 405 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:16:56.111822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc611a51-5ac7-42f4-bf39-a2c3812ee2e9 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:56.125385 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc611a51-5ac7-42f4-bf39-a2c3812ee2e9 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:56.192924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:56.230763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da8976-c1ed-4111-9487-dacb7a564dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:56.231772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da8976-c1ed-4111-9487-dacb7a564dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.232288 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da8976-c1ed-4111-9487-dacb7a564dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.232749 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da8976-c1ed-4111-9487-dacb7a564dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.233903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da8976-c1ed-4111-9487-dacb7a564dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:56.236146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da8976-c1ed-4111-9487-dacb7a564dd2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:56.263635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1142/2273] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:56 2022] POST /identity/v3/users => generated 309 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:16:56.268975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589b4ac-ea71-4a6e-9e7d-33e476dd1177 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:56.286183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589b4ac-ea71-4a6e-9e7d-33e476dd1177 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:56.291583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc6816-9963-478d-9073-265fafaa5afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:56.292216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc6816-9963-478d-9073-265fafaa5afc admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.292506 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc6816-9963-478d-9073-265fafaa5afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.292949 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc6816-9963-478d-9073-265fafaa5afc admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.293660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc6816-9963-478d-9073-265fafaa5afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:56.295331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc6816-9963-478d-9073-265fafaa5afc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:56.317289 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da8976-c1ed-4111-9487-dacb7a564dd2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:56.317477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1132/2274] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:56.336187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da8976-c1ed-4111-9487-dacb7a564dd2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:56.356428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2451c31-b415-43b8-8166-41bc628b0ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:56.357511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2451c31-b415-43b8-8166-41bc628b0ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.358053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2451c31-b415-43b8-8166-41bc628b0ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.359135 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2451c31-b415-43b8-8166-41bc628b0ed8 admin admin] PATH_INFO: `/v3/projects/8035e71c1f4f43c28ce77b0bead14ce2/users/6af68ddcbf1749e081620b05b7bca02f/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.378255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2451c31-b415-43b8-8166-41bc628b0ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8035e71c1f4f43c28ce77b0bead14ce2, user_id=6af68ddcbf1749e081620b05b7bca02f, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:56.381905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2451c31-b415-43b8-8166-41bc628b0ed8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:56.400563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1143/2275] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:56 2022] PUT /identity/v3/projects/8035e71c1f4f43c28ce77b0bead14ce2/users/6af68ddcbf1749e081620b05b7bca02f/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:56.406187 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc6816-9963-478d-9073-265fafaa5afc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:56.421764 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc6816-9963-478d-9073-265fafaa5afc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:56.534505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa701c5-45ce-4726-b6a6-904d9d5c12b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:56.535122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa701c5-45ce-4726-b6a6-904d9d5c12b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.535397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa701c5-45ce-4726-b6a6-904d9d5c12b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.535704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa701c5-45ce-4726-b6a6-904d9d5c12b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.536353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa701c5-45ce-4726-b6a6-904d9d5c12b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:56.537975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa701c5-45ce-4726-b6a6-904d9d5c12b0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:56.568924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2451c31-b415-43b8-8166-41bc628b0ed8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:56.571330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b2451c31-b415-43b8-8166-41bc628b0ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:56.584408 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1133/2276] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:56.585662 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa701c5-45ce-4726-b6a6-904d9d5c12b0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:56.596048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2451c31-b415-43b8-8166-41bc628b0ed8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:56.617551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b83cb-bbf8-4805-b081-faf17f9cccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:56.618206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b83cb-bbf8-4805-b081-faf17f9cccdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.618485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b83cb-bbf8-4805-b081-faf17f9cccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.618755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b83cb-bbf8-4805-b081-faf17f9cccdb admin admin] PATH_INFO: `/v3/projects/183480f0338b40ccb773b94a55d504af` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.620514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b83cb-bbf8-4805-b081-faf17f9cccdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=183480f0338b40ccb773b94a55d504af)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:56.623134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b83cb-bbf8-4805-b081-faf17f9cccdb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:56.624703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa701c5-45ce-4726-b6a6-904d9d5c12b0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:56.640414 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651c730-70f3-4577-bc6f-5b61416e21d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:56.642429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651c730-70f3-4577-bc6f-5b61416e21d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.642809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651c730-70f3-4577-bc6f-5b61416e21d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.643173 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651c730-70f3-4577-bc6f-5b61416e21d2 admin admin] PATH_INFO: `/v3/projects/8035e71c1f4f43c28ce77b0bead14ce2/users/6af68ddcbf1749e081620b05b7bca02f/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.645840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651c730-70f3-4577-bc6f-5b61416e21d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8035e71c1f4f43c28ce77b0bead14ce2, user_id=6af68ddcbf1749e081620b05b7bca02f, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:56.649900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651c730-70f3-4577-bc6f-5b61416e21d2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:56.678525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1134/2277] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:56 2022] PUT /identity/v3/projects/8035e71c1f4f43c28ce77b0bead14ce2/users/6af68ddcbf1749e081620b05b7bca02f/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:56.687504 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b2c50-dec9-48cf-a9cd-4ad1c6c21f27 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.687784 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b2c50-dec9-48cf-a9cd-4ad1c6c21f27 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.688050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b2c50-dec9-48cf-a9cd-4ad1c6c21f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.689953 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a10b83cb-bbf8-4805-b081-faf17f9cccdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '183480f0338b40ccb773b94a55d504af', 'request_id': 'req-a10b83cb-bbf8-4805-b081-faf17f9cccdb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:56.700460 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a10b83cb-bbf8-4805-b081-faf17f9cccdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '183480f0338b40ccb773b94a55d504af', 'request_id': 'req-a10b83cb-bbf8-4805-b081-faf17f9cccdb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:56.708046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a10b83cb-bbf8-4805-b081-faf17f9cccdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '183480f0338b40ccb773b94a55d504af', 'request_id': 'req-a10b83cb-bbf8-4805-b081-faf17f9cccdb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:56.732434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f22b2c50-dec9-48cf-a9cd-4ad1c6c21f27 None None] MFA Rules not processed for user `6af68ddcbf1749e081620b05b7bca02f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:56.733046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a10b83cb-bbf8-4805-b081-faf17f9cccdb admin admin] The token cache is being invalidate because project 183480f0338b40ccb773b94a55d504af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:56.733405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a10b83cb-bbf8-4805-b081-faf17f9cccdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:56.734084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f22b2c50-dec9-48cf-a9cd-4ad1c6c21f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:56.737517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a10b83cb-bbf8-4805-b081-faf17f9cccdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:56.739617 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1144/2278] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:56 2022] DELETE /identity/v3/projects/183480f0338b40ccb773b94a55d504af => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:56.745384 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6d621-b4ab-4ea7-8ae2-f2390e38cd5a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.745662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6d621-b4ab-4ea7-8ae2-f2390e38cd5a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.746106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6d621-b4ab-4ea7-8ae2-f2390e38cd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.768710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6de6d621-b4ab-4ea7-8ae2-f2390e38cd5a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:56.779575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6de6d621-b4ab-4ea7-8ae2-f2390e38cd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:56.846443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1135/2279] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:56.864390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1145/2280] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:56.866063 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af07af-9e89-47fd-b335-8ba2e4718e0e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.866356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af07af-9e89-47fd-b335-8ba2e4718e0e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.866633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af07af-9e89-47fd-b335-8ba2e4718e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.871341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6d621-b4ab-4ea7-8ae2-f2390e38cd5a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:56.872948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6de6d621-b4ab-4ea7-8ae2-f2390e38cd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:56.888879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-17af07af-9e89-47fd-b335-8ba2e4718e0e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:56.890108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-17af07af-9e89-47fd-b335-8ba2e4718e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:56.899122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1136/2281] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:56.903871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af07af-9e89-47fd-b335-8ba2e4718e0e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:56.917393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af07af-9e89-47fd-b335-8ba2e4718e0e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:56.925330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:56.927561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d132-fbdd-4a9c-9a86-b38229b7111c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:56.928185 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d132-fbdd-4a9c-9a86-b38229b7111c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.928456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d132-fbdd-4a9c-9a86-b38229b7111c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.928751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d132-fbdd-4a9c-9a86-b38229b7111c admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.929406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d132-fbdd-4a9c-9a86-b38229b7111c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:16:56.929702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d132-fbdd-4a9c-9a86-b38229b7111c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:56.931442 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d132-fbdd-4a9c-9a86-b38229b7111c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:56.957169 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1137/2282] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:16:56.960352 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6d621-b4ab-4ea7-8ae2-f2390e38cd5a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:56.961440 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d132-fbdd-4a9c-9a86-b38229b7111c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:56.965633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d8cee-b960-4d5f-b01c-3b7047be0cde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:56.965871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d8cee-b960-4d5f-b01c-3b7047be0cde service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.966151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d8cee-b960-4d5f-b01c-3b7047be0cde service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.966317 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d8cee-b960-4d5f-b01c-3b7047be0cde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.972806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d132-fbdd-4a9c-9a86-b38229b7111c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:56.976331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d8cee-b960-4d5f-b01c-3b7047be0cde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:56.978616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d8cee-b960-4d5f-b01c-3b7047be0cde service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:56.983510 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e3439-c069-4b3b-b43d-29850e0ea1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:56.984130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e3439-c069-4b3b-b43d-29850e0ea1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:56.985044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e3439-c069-4b3b-b43d-29850e0ea1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:56.985044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e3439-c069-4b3b-b43d-29850e0ea1d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:56.985320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e3439-c069-4b3b-b43d-29850e0ea1d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:56.987222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e3439-c069-4b3b-b43d-29850e0ea1d9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:56.989792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1146/2283] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:56 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:56.992689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:57.021446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1138/2284] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:56 2022] POST /identity/v3/projects => generated 397 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:16:57.025752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d8cee-b960-4d5f-b01c-3b7047be0cde service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:57.062127 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d8cee-b960-4d5f-b01c-3b7047be0cde service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:57.165938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d632c5-7783-485c-abc9-cf4d52bd9b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:57.168749 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d632c5-7783-485c-abc9-cf4d52bd9b0b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:57.171141 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d632c5-7783-485c-abc9-cf4d52bd9b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:57.178949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d632c5-7783-485c-abc9-cf4d52bd9b0b admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:57.180002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d632c5-7783-485c-abc9-cf4d52bd9b0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:57.182802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d632c5-7783-485c-abc9-cf4d52bd9b0b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:57.221195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1147/2285] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:57 2022] POST /identity/v3/users => generated 299 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:16:57.225737 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e3439-c069-4b3b-b43d-29850e0ea1d9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:57.238578 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e3439-c069-4b3b-b43d-29850e0ea1d9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:57.248602 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a29ea-e68d-412d-9de9-45dff4fd7a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:57.249249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a29ea-e68d-412d-9de9-45dff4fd7a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:57.249388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d632c5-7783-485c-abc9-cf4d52bd9b0b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:57.249533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a29ea-e68d-412d-9de9-45dff4fd7a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:57.249778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a29ea-e68d-412d-9de9-45dff4fd7a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:57.250752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a29ea-e68d-412d-9de9-45dff4fd7a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:57.251402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f3d632c5-7783-485c-abc9-cf4d52bd9b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:57.252703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a29ea-e68d-412d-9de9-45dff4fd7a7a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:57.264715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1139/2286] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:16:57.272959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a29ea-e68d-412d-9de9-45dff4fd7a7a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:57.286620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a29ea-e68d-412d-9de9-45dff4fd7a7a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:57.291601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d207a-928c-4d9d-aa04-f4492cc08d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:57.292206 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d207a-928c-4d9d-aa04-f4492cc08d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:57.292479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d207a-928c-4d9d-aa04-f4492cc08d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:57.293104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d207a-928c-4d9d-aa04-f4492cc08d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:57.293762 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d207a-928c-4d9d-aa04-f4492cc08d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:57.295374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d207a-928c-4d9d-aa04-f4492cc08d20 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:57.309615 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1140/2287] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:16:57.311297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d207a-928c-4d9d-aa04-f4492cc08d20 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:57.322871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d207a-928c-4d9d-aa04-f4492cc08d20 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:57.326173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d632c5-7783-485c-abc9-cf4d52bd9b0b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:57.328430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9d007-9188-4721-beab-f7d15e8c587b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:57.329314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9d007-9188-4721-beab-f7d15e8c587b admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:57.329730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9d007-9188-4721-beab-f7d15e8c587b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:57.330262 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208cac0-9d2d-4cce-b635-da865a39fa9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:57.330825 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208cac0-9d2d-4cce-b635-da865a39fa9f service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:57.331158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208cac0-9d2d-4cce-b635-da865a39fa9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:57.331538 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208cac0-9d2d-4cce-b635-da865a39fa9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:57.333603 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8208cac0-9d2d-4cce-b635-da865a39fa9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:57.335936 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9d007-9188-4721-beab-f7d15e8c587b admin admin] PATH_INFO: `/v3/projects/d376babf9c5d4604a5674fd3cb141c9e/users/b16b2dd9478348a593868ffeb24214ce/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:57.353220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9d007-9188-4721-beab-f7d15e8c587b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d376babf9c5d4604a5674fd3cb141c9e, user_id=b16b2dd9478348a593868ffeb24214ce, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:57.356016 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9d007-9188-4721-beab-f7d15e8c587b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:57.380733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1141/2288] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:57 2022] PUT /identity/v3/projects/d376babf9c5d4604a5674fd3cb141c9e/users/b16b2dd9478348a593868ffeb24214ce/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:57.383074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590983a3-2c67-4109-a7e6-f160c5307eff None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:57.383550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590983a3-2c67-4109-a7e6-f160c5307eff None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:57.384026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590983a3-2c67-4109-a7e6-f160c5307eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:57.410868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208cac0-9d2d-4cce-b635-da865a39fa9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:57.412564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208cac0-9d2d-4cce-b635-da865a39fa9f service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:57.422376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-590983a3-2c67-4109-a7e6-f160c5307eff None None] MFA Rules not processed for user `b16b2dd9478348a593868ffeb24214ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:57.424002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1148/2289] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:57 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:57.424129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-590983a3-2c67-4109-a7e6-f160c5307eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:57.507160 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1142/2290] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:57 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:57.646080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208cac0-9d2d-4cce-b635-da865a39fa9f service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:57.655554 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208cac0-9d2d-4cce-b635-da865a39fa9f service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:57.672831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:57.708667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ec17f-4156-4413-8c46-e593ef6764fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:57.709293 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ec17f-4156-4413-8c46-e593ef6764fa service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:57.709564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ec17f-4156-4413-8c46-e593ef6764fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:57.709832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ec17f-4156-4413-8c46-e593ef6764fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:57.719520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ec17f-4156-4413-8c46-e593ef6764fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:57.720956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ec17f-4156-4413-8c46-e593ef6764fa service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:57.731884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1149/2291] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:57 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:58.696463 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590983a3-2c67-4109-a7e6-f160c5307eff None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:58.698032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-590983a3-2c67-4109-a7e6-f160c5307eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:58.785182 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590983a3-2c67-4109-a7e6-f160c5307eff None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:58.789683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:58.790321 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:58.790603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:58.790874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] PATH_INFO: `/v3/users/8dd7bb424a25489db00059483f558b9a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:58.792430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dd7bb424a25489db00059483f558b9a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:58.794543 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:58.834696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:58.851126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dd7bb424a25489db00059483f558b9a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:58.851780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dd7bb424a25489db00059483f558b9a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:58.863219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dd7bb424a25489db00059483f558b9a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:58.876171 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dd7bb424a25489db00059483f558b9a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:58.877227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dd7bb424a25489db00059483f558b9a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:58.895704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1143/2292] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:16:58 2022] DELETE /identity/v3/users/8dd7bb424a25489db00059483f558b9a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:58.901283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab721bf8-3e56-4ca7-a8fa-068b8ce9f56a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:58.901808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab721bf8-3e56-4ca7-a8fa-068b8ce9f56a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:58.902027 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab721bf8-3e56-4ca7-a8fa-068b8ce9f56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:58.924413 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ab721bf8-3e56-4ca7-a8fa-068b8ce9f56a None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:58.925904 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ab721bf8-3e56-4ca7-a8fa-068b8ce9f56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:58.993228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1150/2293] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:16:59.003349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:59.005077 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:59.078876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d318a-d9e6-4c5f-a184-5c96bebd4f46 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:59.083234 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d400d3d-a0bc-4d4f-bac2-98dbba4b8728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:59.083841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d400d3d-a0bc-4d4f-bac2-98dbba4b8728 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:59.084127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d400d3d-a0bc-4d4f-bac2-98dbba4b8728 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:59.084389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d400d3d-a0bc-4d4f-bac2-98dbba4b8728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:59.094417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d400d3d-a0bc-4d4f-bac2-98dbba4b8728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:59.095898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d400d3d-a0bc-4d4f-bac2-98dbba4b8728 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:59.106222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1144/2294] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:16:59.307039 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab721bf8-3e56-4ca7-a8fa-068b8ce9f56a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:59.308752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ab721bf8-3e56-4ca7-a8fa-068b8ce9f56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:59.322065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab721bf8-3e56-4ca7-a8fa-068b8ce9f56a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:59.326748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfaf59a-b635-4580-9b09-9c007d00bf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:59.327405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfaf59a-b635-4580-9b09-9c007d00bf8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:59.327688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfaf59a-b635-4580-9b09-9c007d00bf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:59.327964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfaf59a-b635-4580-9b09-9c007d00bf8c admin admin] PATH_INFO: `/v3/projects/acf9183eeeef40f7a99a49d0abfeffa6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:59.329209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfaf59a-b635-4580-9b09-9c007d00bf8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=acf9183eeeef40f7a99a49d0abfeffa6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:59.331133 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfaf59a-b635-4580-9b09-9c007d00bf8c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:59.387992 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9dfaf59a-b635-4580-9b09-9c007d00bf8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acf9183eeeef40f7a99a49d0abfeffa6', 'request_id': 'req-9dfaf59a-b635-4580-9b09-9c007d00bf8c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:59.394360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9dfaf59a-b635-4580-9b09-9c007d00bf8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acf9183eeeef40f7a99a49d0abfeffa6', 'request_id': 'req-9dfaf59a-b635-4580-9b09-9c007d00bf8c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:59.400626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9dfaf59a-b635-4580-9b09-9c007d00bf8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acf9183eeeef40f7a99a49d0abfeffa6', 'request_id': 'req-9dfaf59a-b635-4580-9b09-9c007d00bf8c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:59.414184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9dfaf59a-b635-4580-9b09-9c007d00bf8c admin admin] The token cache is being invalidate because project acf9183eeeef40f7a99a49d0abfeffa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:16:59.414468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9dfaf59a-b635-4580-9b09-9c007d00bf8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:59.415065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9dfaf59a-b635-4580-9b09-9c007d00bf8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:16:59.416586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1151/2295] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:16:59 2022] DELETE /identity/v3/projects/acf9183eeeef40f7a99a49d0abfeffa6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:59.422067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f77318-7367-4efc-bd67-4401495213d6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:59.422388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f77318-7367-4efc-bd67-4401495213d6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:59.422679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f77318-7367-4efc-bd67-4401495213d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:59.450151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-27f77318-7367-4efc-bd67-4401495213d6 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:59.451477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-27f77318-7367-4efc-bd67-4401495213d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:59.521951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1145/2296] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:59.530468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f107e-10e7-47b2-8673-b5067ad003db None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:59.530707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f107e-10e7-47b2-8673-b5067ad003db None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:59.531018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f107e-10e7-47b2-8673-b5067ad003db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:59.553533 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5b2f107e-10e7-47b2-8673-b5067ad003db None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:59.554666 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5b2f107e-10e7-47b2-8673-b5067ad003db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:59.563684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1152/2297] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:16:59.568055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f77318-7367-4efc-bd67-4401495213d6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:59.578567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f77318-7367-4efc-bd67-4401495213d6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:59.584408 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3533f5-e086-445d-8ab3-1120affae639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:59.585065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3533f5-e086-445d-8ab3-1120affae639 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:59.585367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3533f5-e086-445d-8ab3-1120affae639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:59.585636 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3533f5-e086-445d-8ab3-1120affae639 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:59.586281 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3533f5-e086-445d-8ab3-1120affae639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:16:59.586584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3533f5-e086-445d-8ab3-1120affae639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:59.588229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3533f5-e086-445d-8ab3-1120affae639 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:59.603077 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1146/2298] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:16:59.607861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f107e-10e7-47b2-8673-b5067ad003db None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:59.617863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f107e-10e7-47b2-8673-b5067ad003db None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:59.624903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0133789-a90c-4ad3-8d25-9dd6c3fad333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:59.625584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0133789-a90c-4ad3-8d25-9dd6c3fad333 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:59.625860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0133789-a90c-4ad3-8d25-9dd6c3fad333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:59.626145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0133789-a90c-4ad3-8d25-9dd6c3fad333 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:59.626813 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0133789-a90c-4ad3-8d25-9dd6c3fad333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:59.628796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0133789-a90c-4ad3-8d25-9dd6c3fad333 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:59.659115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1153/2299] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:16:59 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:59.663633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3533f5-e086-445d-8ab3-1120affae639 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:59.675131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3533f5-e086-445d-8ab3-1120affae639 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:59.716898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:59.743514 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493dc6e-f8bf-469f-85f4-58bea849d7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:59.743740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493dc6e-f8bf-469f-85f4-58bea849d7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:59.743863 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493dc6e-f8bf-469f-85f4-58bea849d7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:59.744096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493dc6e-f8bf-469f-85f4-58bea849d7c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:59.744770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493dc6e-f8bf-469f-85f4-58bea849d7c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:59.746711 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493dc6e-f8bf-469f-85f4-58bea849d7c4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:59.772764 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1147/2300] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:16:59 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:16:59.775042 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0133789-a90c-4ad3-8d25-9dd6c3fad333 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:59.785226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0133789-a90c-4ad3-8d25-9dd6c3fad333 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:59.790064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03023-a9e3-48ed-a015-d9ef142c2efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:59.790671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03023-a9e3-48ed-a015-d9ef142c2efc admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:59.790957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03023-a9e3-48ed-a015-d9ef142c2efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:59.791219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03023-a9e3-48ed-a015-d9ef142c2efc admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:59.791868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a03023-a9e3-48ed-a015-d9ef142c2efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:59.793618 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a03023-a9e3-48ed-a015-d9ef142c2efc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:59.804979 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1154/2301] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:59.808876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493dc6e-f8bf-469f-85f4-58bea849d7c4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:59.819251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493dc6e-f8bf-469f-85f4-58bea849d7c4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:59.823993 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d385a9-27b3-4ac2-8be9-58f530f104a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:59.824593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d385a9-27b3-4ac2-8be9-58f530f104a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:59.824901 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d385a9-27b3-4ac2-8be9-58f530f104a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:59.825169 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d385a9-27b3-4ac2-8be9-58f530f104a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:59.825805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d385a9-27b3-4ac2-8be9-58f530f104a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:59.827408 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d385a9-27b3-4ac2-8be9-58f530f104a6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:59.837453 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1148/2302] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:16:59.841508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03023-a9e3-48ed-a015-d9ef142c2efc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:16:59.851584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03023-a9e3-48ed-a015-d9ef142c2efc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:16:59.856346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb7b84-afc1-4519-89b4-c6fb155d8304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:16:59.857022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb7b84-afc1-4519-89b4-c6fb155d8304 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:59.857304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb7b84-afc1-4519-89b4-c6fb155d8304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:59.857571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb7b84-afc1-4519-89b4-c6fb155d8304 admin admin] PATH_INFO: `/v3/projects/5801984f41a043daa52dbef9de89df9a/users/6d0c1c510b0a4fa39927f7f92352d034/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:59.859739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:16:59.875807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb7b84-afc1-4519-89b4-c6fb155d8304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5801984f41a043daa52dbef9de89df9a, user_id=6d0c1c510b0a4fa39927f7f92352d034, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:16:59.878549 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb7b84-afc1-4519-89b4-c6fb155d8304 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:16:59.895874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1155/2303] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:16:59 2022] PUT /identity/v3/projects/5801984f41a043daa52dbef9de89df9a/users/6d0c1c510b0a4fa39927f7f92352d034/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:16:59.901083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45ddb3-b85e-48bf-b1bb-5418b7db7362 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:16:59.901371 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45ddb3-b85e-48bf-b1bb-5418b7db7362 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:16:59.901648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45ddb3-b85e-48bf-b1bb-5418b7db7362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:16:59.928125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7c45ddb3-b85e-48bf-b1bb-5418b7db7362 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:16:59.929994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7c45ddb3-b85e-48bf-b1bb-5418b7db7362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:16:59.998600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1149/2304] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:17:00.009699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb7b84-afc1-4519-89b4-c6fb155d8304 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:00.011663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9feb7b84-afc1-4519-89b4-c6fb155d8304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:00.079600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb7b84-afc1-4519-89b4-c6fb155d8304 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:00.083788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e39ab8-98c7-46bd-9435-2c2cf4a8ad7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:00.084466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e39ab8-98c7-46bd-9435-2c2cf4a8ad7b service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:00.084773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e39ab8-98c7-46bd-9435-2c2cf4a8ad7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:00.085070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e39ab8-98c7-46bd-9435-2c2cf4a8ad7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:00.095652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e39ab8-98c7-46bd-9435-2c2cf4a8ad7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:00.097158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e39ab8-98c7-46bd-9435-2c2cf4a8ad7b service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:00.107442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1156/2305] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:17:04.314655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe57170-5dc6-48d9-958f-0aec5769ad07 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:04.315080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe57170-5dc6-48d9-958f-0aec5769ad07 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:04.315185 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe57170-5dc6-48d9-958f-0aec5769ad07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:04.346585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1fe57170-5dc6-48d9-958f-0aec5769ad07 None None] MFA Rules not processed for user `6d0c1c510b0a4fa39927f7f92352d034`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:04.347855 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1fe57170-5dc6-48d9-958f-0aec5769ad07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:04.426659 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1150/2306] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:04.431598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e39ab8-98c7-46bd-9435-2c2cf4a8ad7b service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:04.442008 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e39ab8-98c7-46bd-9435-2c2cf4a8ad7b service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:04.446902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cac6ab-cf67-4b52-92d1-f016ff27dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:04.447656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac6ab-cf67-4b52-92d1-f016ff27dcd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:04.447781 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac6ab-cf67-4b52-92d1-f016ff27dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:04.448052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac6ab-cf67-4b52-92d1-f016ff27dcd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:04.448756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cac6ab-cf67-4b52-92d1-f016ff27dcd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:04.451346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cac6ab-cf67-4b52-92d1-f016ff27dcd8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:04.481430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1157/2307] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:17:04 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:17:04.485656 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe57170-5dc6-48d9-958f-0aec5769ad07 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:04.497206 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe57170-5dc6-48d9-958f-0aec5769ad07 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:04.544782 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:04.559526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521739e-7282-44be-b79a-ba9b49637823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:04.560770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521739e-7282-44be-b79a-ba9b49637823 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:04.560770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521739e-7282-44be-b79a-ba9b49637823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:04.561942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521739e-7282-44be-b79a-ba9b49637823 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:04.562627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521739e-7282-44be-b79a-ba9b49637823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:04.564673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521739e-7282-44be-b79a-ba9b49637823 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:04.598752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1151/2308] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:17:04 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:17:04.603006 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cac6ab-cf67-4b52-92d1-f016ff27dcd8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:04.614635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cac6ab-cf67-4b52-92d1-f016ff27dcd8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:04.619808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c376c4-f97e-4993-91c6-57c493944bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:04.620594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c376c4-f97e-4993-91c6-57c493944bbf admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:04.620905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c376c4-f97e-4993-91c6-57c493944bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:04.621182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c376c4-f97e-4993-91c6-57c493944bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:04.622037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c376c4-f97e-4993-91c6-57c493944bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:04.625686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c376c4-f97e-4993-91c6-57c493944bbf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:04.638268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1158/2309] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:17:04.643456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521739e-7282-44be-b79a-ba9b49637823 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:04.657816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521739e-7282-44be-b79a-ba9b49637823 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:04.667577 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657eb7f-6c9e-450c-bb08-1b0737ddf448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:04.668487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657eb7f-6c9e-450c-bb08-1b0737ddf448 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:04.668954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657eb7f-6c9e-450c-bb08-1b0737ddf448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:04.669315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657eb7f-6c9e-450c-bb08-1b0737ddf448 admin admin] PATH_INFO: `/v3/projects/a6e7246e3e3247bfa91d9122dbb59226/users/d782519572b34fadbbab114d96152eda/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:04.693160 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657eb7f-6c9e-450c-bb08-1b0737ddf448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e7246e3e3247bfa91d9122dbb59226, user_id=d782519572b34fadbbab114d96152eda, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:04.695972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657eb7f-6c9e-450c-bb08-1b0737ddf448 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:04.715718 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1152/2310] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:17:04 2022] PUT /identity/v3/projects/a6e7246e3e3247bfa91d9122dbb59226/users/d782519572b34fadbbab114d96152eda/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:04.716898 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c376c4-f97e-4993-91c6-57c493944bbf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:04.727834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c376c4-f97e-4993-91c6-57c493944bbf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:04.786890 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3030aa8-16cc-4908-a31c-54c2f6a1285d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:04.787527 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3030aa8-16cc-4908-a31c-54c2f6a1285d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:04.787808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3030aa8-16cc-4908-a31c-54c2f6a1285d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:04.788084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3030aa8-16cc-4908-a31c-54c2f6a1285d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:04.788767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3030aa8-16cc-4908-a31c-54c2f6a1285d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:04.790400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3030aa8-16cc-4908-a31c-54c2f6a1285d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:04.806614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1159/2311] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:17:04.812707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657eb7f-6c9e-450c-bb08-1b0737ddf448 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:04.841232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657eb7f-6c9e-450c-bb08-1b0737ddf448 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:04.846516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e14e1-b276-42f7-a1cf-4e5783146675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:04.847200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e14e1-b276-42f7-a1cf-4e5783146675 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:04.847479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e14e1-b276-42f7-a1cf-4e5783146675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:04.847751 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e14e1-b276-42f7-a1cf-4e5783146675 admin admin] PATH_INFO: `/v3/projects/a6e7246e3e3247bfa91d9122dbb59226/users/d782519572b34fadbbab114d96152eda/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:04.850296 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e14e1-b276-42f7-a1cf-4e5783146675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e7246e3e3247bfa91d9122dbb59226, user_id=d782519572b34fadbbab114d96152eda, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:04.853046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e14e1-b276-42f7-a1cf-4e5783146675 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:04.864019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1153/2312] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:17:04 2022] PUT /identity/v3/projects/a6e7246e3e3247bfa91d9122dbb59226/users/d782519572b34fadbbab114d96152eda/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:09.922440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f0fd-12fa-4d94-b25d-476b86b2549d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:09.923031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f0fd-12fa-4d94-b25d-476b86b2549d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:09.923031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f0fd-12fa-4d94-b25d-476b86b2549d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:09.955797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6404f0fd-12fa-4d94-b25d-476b86b2549d None None] MFA Rules not processed for user `d782519572b34fadbbab114d96152eda`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:09.957067 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6404f0fd-12fa-4d94-b25d-476b86b2549d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:10.002675 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:10.047396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1160/2313] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:09 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:10.061497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e14e1-b276-42f7-a1cf-4e5783146675 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:10.064868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-011e14e1-b276-42f7-a1cf-4e5783146675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:10.121431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a672df-3542-49a0-8b51-9aee45ccd017 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:10.121726 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a672df-3542-49a0-8b51-9aee45ccd017 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:10.121999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a672df-3542-49a0-8b51-9aee45ccd017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:10.146441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b7a672df-3542-49a0-8b51-9aee45ccd017 None None] MFA Rules not processed for user `2ec47d6e2e9b40a6bd207c084041f7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:10.147696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b7a672df-3542-49a0-8b51-9aee45ccd017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:10.165576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e14e1-b276-42f7-a1cf-4e5783146675 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:10.173099 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c89099-b76f-423d-91c0-49abaada7a69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:10.174095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c89099-b76f-423d-91c0-49abaada7a69 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:10.174374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c89099-b76f-423d-91c0-49abaada7a69 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:10.174646 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c89099-b76f-423d-91c0-49abaada7a69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:10.220209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1161/2314] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:17:10.251686 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c89099-b76f-423d-91c0-49abaada7a69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:10.253568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c89099-b76f-423d-91c0-49abaada7a69 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:10.263954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1154/2315] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:17:12.758256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a672df-3542-49a0-8b51-9aee45ccd017 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:12.759686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b7a672df-3542-49a0-8b51-9aee45ccd017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:12.836124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a672df-3542-49a0-8b51-9aee45ccd017 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:12.842153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4da47-5223-4b6d-9c43-b1dc511f792b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:12.842847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4da47-5223-4b6d-9c43-b1dc511f792b service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:12.843162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4da47-5223-4b6d-9c43-b1dc511f792b service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:12.843479 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4da47-5223-4b6d-9c43-b1dc511f792b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:12.845681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d1a4da47-5223-4b6d-9c43-b1dc511f792b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:12.859511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4da47-5223-4b6d-9c43-b1dc511f792b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:12.861235 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4da47-5223-4b6d-9c43-b1dc511f792b service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:12.872052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1162/2316] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:17:15.925981 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c89099-b76f-423d-91c0-49abaada7a69 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:15.927543 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d7c89099-b76f-423d-91c0-49abaada7a69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:15.978212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:16.004548 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c89099-b76f-423d-91c0-49abaada7a69 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:16.009417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:16.010163 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:16.010446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:16.010721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] PATH_INFO: `/v3/users/17aa0d27da234aeb957e1ef96fe61413` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:16.011884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17aa0d27da234aeb957e1ef96fe61413)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:16.013895 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:16.066820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17aa0d27da234aeb957e1ef96fe61413'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.067550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17aa0d27da234aeb957e1ef96fe61413'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.074439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17aa0d27da234aeb957e1ef96fe61413'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.086118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17aa0d27da234aeb957e1ef96fe61413'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.086793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17aa0d27da234aeb957e1ef96fe61413'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.101866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1155/2317] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:17:15 2022] DELETE /identity/v3/users/17aa0d27da234aeb957e1ef96fe61413 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:16.106601 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15032405-029a-40f0-8fd2-ebc0a6c9f7c2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:16.107141 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15032405-029a-40f0-8fd2-ebc0a6c9f7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:16.107224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15032405-029a-40f0-8fd2-ebc0a6c9f7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:16.127595 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-15032405-029a-40f0-8fd2-ebc0a6c9f7c2 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:16.128877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-15032405-029a-40f0-8fd2-ebc0a6c9f7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:16.191178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1163/2318] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:17:16.201713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:16.203378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:16.270349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54148795-f124-4bdb-b984-0fbda8300ce2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:16.274092 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1eef4-491a-47d7-89d2-928c838b54bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:16.274703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1eef4-491a-47d7-89d2-928c838b54bb service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:16.274979 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1eef4-491a-47d7-89d2-928c838b54bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:16.275246 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1eef4-491a-47d7-89d2-928c838b54bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:16.287021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1eef4-491a-47d7-89d2-928c838b54bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:16.288471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1eef4-491a-47d7-89d2-928c838b54bb service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:16.299448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1156/2319] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:17:16.366781 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15032405-029a-40f0-8fd2-ebc0a6c9f7c2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:16.368849 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-15032405-029a-40f0-8fd2-ebc0a6c9f7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:16.382320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15032405-029a-40f0-8fd2-ebc0a6c9f7c2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:16.387689 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:16.388530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:16.388846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:16.389121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] PATH_INFO: `/v3/users/b16b2dd9478348a593868ffeb24214ce` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:16.390263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b16b2dd9478348a593868ffeb24214ce)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:16.392796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:16.395479 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:16.464982 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b16b2dd9478348a593868ffeb24214ce'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.465751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b16b2dd9478348a593868ffeb24214ce'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.474585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b16b2dd9478348a593868ffeb24214ce'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.512479 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b16b2dd9478348a593868ffeb24214ce'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.517259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b16b2dd9478348a593868ffeb24214ce'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.546294 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1164/2320] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:17:16 2022] DELETE /identity/v3/users/b16b2dd9478348a593868ffeb24214ce => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:16.551330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce458bb2-2f18-4c1f-9fc3-715c37965fdd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:16.551615 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce458bb2-2f18-4c1f-9fc3-715c37965fdd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:16.552112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce458bb2-2f18-4c1f-9fc3-715c37965fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:16.573066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ce458bb2-2f18-4c1f-9fc3-715c37965fdd None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:16.574378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ce458bb2-2f18-4c1f-9fc3-715c37965fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:16.655334 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:16.683944 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1157/2321] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:16.698444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:16.700018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:16.710437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce458bb2-2f18-4c1f-9fc3-715c37965fdd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:16.712017 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ce458bb2-2f18-4c1f-9fc3-715c37965fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:16.733784 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce458bb2-2f18-4c1f-9fc3-715c37965fdd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:16.747018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c8821-dd80-4477-8a94-0e5af095095f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:16.747865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c8821-dd80-4477-8a94-0e5af095095f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:16.748638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c8821-dd80-4477-8a94-0e5af095095f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:16.748975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c8821-dd80-4477-8a94-0e5af095095f admin admin] PATH_INFO: `/v3/projects/70e949fdb7c14e6993449522f16ba84e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:16.750706 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c8821-dd80-4477-8a94-0e5af095095f admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e949fdb7c14e6993449522f16ba84e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:16.752637 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c8821-dd80-4477-8a94-0e5af095095f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:16.794852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29063e-e2b4-4496-913b-1ce81d34fdbc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:16.799374 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed027f83-d964-4a86-8601-8368c92b6848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:16.800020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed027f83-d964-4a86-8601-8368c92b6848 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:16.800228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed027f83-d964-4a86-8601-8368c92b6848 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:16.800572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed027f83-d964-4a86-8601-8368c92b6848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:16.811049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed027f83-d964-4a86-8601-8368c92b6848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:16.812545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed027f83-d964-4a86-8601-8368c92b6848 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:16.823010 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1165/2322] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:17:16.823432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed027f83-d964-4a86-8601-8368c92b6848 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:16.824814 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ed027f83-d964-4a86-8601-8368c92b6848 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:16.832693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-581c8821-dd80-4477-8a94-0e5af095095f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e949fdb7c14e6993449522f16ba84e', 'request_id': 'req-581c8821-dd80-4477-8a94-0e5af095095f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.839484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed027f83-d964-4a86-8601-8368c92b6848 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:16.845203 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-581c8821-dd80-4477-8a94-0e5af095095f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e949fdb7c14e6993449522f16ba84e', 'request_id': 'req-581c8821-dd80-4477-8a94-0e5af095095f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.855095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-581c8821-dd80-4477-8a94-0e5af095095f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e949fdb7c14e6993449522f16ba84e', 'request_id': 'req-581c8821-dd80-4477-8a94-0e5af095095f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.867958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-581c8821-dd80-4477-8a94-0e5af095095f admin admin] The token cache is being invalidate because project 70e949fdb7c14e6993449522f16ba84e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:17:16.868344 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-581c8821-dd80-4477-8a94-0e5af095095f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.869327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-581c8821-dd80-4477-8a94-0e5af095095f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:16.872720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1158/2323] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:17:16 2022] DELETE /identity/v3/projects/70e949fdb7c14e6993449522f16ba84e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:16.876542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ecbd6-4185-470b-9c9e-dbd46d2d7e2f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:16.877083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ecbd6-4185-470b-9c9e-dbd46d2d7e2f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:16.877263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ecbd6-4185-470b-9c9e-dbd46d2d7e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:16.903961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-535ecbd6-4185-470b-9c9e-dbd46d2d7e2f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:16.907543 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-535ecbd6-4185-470b-9c9e-dbd46d2d7e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:16.933495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d724eb9-ad81-4d6e-8f3a-1330f6781dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:16.934364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d724eb9-ad81-4d6e-8f3a-1330f6781dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:16.934640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d724eb9-ad81-4d6e-8f3a-1330f6781dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:16.936019 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d724eb9-ad81-4d6e-8f3a-1330f6781dd7 admin admin] PATH_INFO: `/v3/users/cc4824e9629d4f1caf04a7270ea73ba0` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:16.938827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d724eb9-ad81-4d6e-8f3a-1330f6781dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc4824e9629d4f1caf04a7270ea73ba0)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:16.941137 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d724eb9-ad81-4d6e-8f3a-1330f6781dd7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:16.996351 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:17.026319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9d724eb9-ad81-4d6e-8f3a-1330f6781dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc4824e9629d4f1caf04a7270ea73ba0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:17.027112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9d724eb9-ad81-4d6e-8f3a-1330f6781dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc4824e9629d4f1caf04a7270ea73ba0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:17.028689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1159/2324] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:17:17.037721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9d724eb9-ad81-4d6e-8f3a-1330f6781dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc4824e9629d4f1caf04a7270ea73ba0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:17.038520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9420b-d3da-4399-a014-8f3e79535bad None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:17.038907 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9420b-d3da-4399-a014-8f3e79535bad None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:17.039194 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9420b-d3da-4399-a014-8f3e79535bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:17.053884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9d724eb9-ad81-4d6e-8f3a-1330f6781dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc4824e9629d4f1caf04a7270ea73ba0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:17.055694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9d724eb9-ad81-4d6e-8f3a-1330f6781dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc4824e9629d4f1caf04a7270ea73ba0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:17.062834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cee9420b-d3da-4399-a014-8f3e79535bad None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:17.064149 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cee9420b-d3da-4399-a014-8f3e79535bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:17.074380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1160/2325] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:17.074701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1166/2326] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:17:16 2022] DELETE /identity/v3/users/cc4824e9629d4f1caf04a7270ea73ba0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:17.078631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9420b-d3da-4399-a014-8f3e79535bad None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:17.082234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f4680-7115-43e9-936b-8a1dd0d6225d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:17.082606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f4680-7115-43e9-936b-8a1dd0d6225d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:17.082969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f4680-7115-43e9-936b-8a1dd0d6225d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:17.088205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9420b-d3da-4399-a014-8f3e79535bad None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:17.103030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ed3f4680-7115-43e9-936b-8a1dd0d6225d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:17.104263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ed3f4680-7115-43e9-936b-8a1dd0d6225d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:17.162961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75190a8-4e83-4594-be01-e0c4b3f13963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:17.163666 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75190a8-4e83-4594-be01-e0c4b3f13963 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:17.163946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75190a8-4e83-4594-be01-e0c4b3f13963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:17.164407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75190a8-4e83-4594-be01-e0c4b3f13963 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:17.165117 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75190a8-4e83-4594-be01-e0c4b3f13963 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:17:17.165423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75190a8-4e83-4594-be01-e0c4b3f13963 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:17.167099 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75190a8-4e83-4594-be01-e0c4b3f13963 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:17.183538 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1167/2327] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:17.192026 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f4680-7115-43e9-936b-8a1dd0d6225d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:17.193883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ed3f4680-7115-43e9-936b-8a1dd0d6225d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:17.200104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1161/2328] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:17:17.200104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75190a8-4e83-4594-be01-e0c4b3f13963 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:17.203165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f75190a8-4e83-4594-be01-e0c4b3f13963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:17.219216 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f4680-7115-43e9-936b-8a1dd0d6225d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:17.221155 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:17.226284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:17.227263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:17.227577 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:17.227940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] PATH_INFO: `/v3/projects/d376babf9c5d4604a5674fd3cb141c9e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:17.229144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d376babf9c5d4604a5674fd3cb141c9e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:17.231099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:17.277405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75190a8-4e83-4594-be01-e0c4b3f13963 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:17.280426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd376babf9c5d4604a5674fd3cb141c9e', 'request_id': 'req-9c20a4b8-4f05-439b-9317-47fe50c57d72'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:17.286878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd376babf9c5d4604a5674fd3cb141c9e', 'request_id': 'req-9c20a4b8-4f05-439b-9317-47fe50c57d72'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:17.294059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd376babf9c5d4604a5674fd3cb141c9e', 'request_id': 'req-9c20a4b8-4f05-439b-9317-47fe50c57d72'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:17.308153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] The token cache is being invalidate because project d376babf9c5d4604a5674fd3cb141c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:17:17.308444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:17.309114 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:17.311056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1168/2329] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:17:17 2022] DELETE /identity/v3/projects/d376babf9c5d4604a5674fd3cb141c9e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:17:17.311629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:17.312994 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:17.345213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bc16f-6139-4ec2-8042-7cfa04d6a733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:17.346012 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bc16f-6139-4ec2-8042-7cfa04d6a733 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:17.346290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bc16f-6139-4ec2-8042-7cfa04d6a733 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:17.346559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bc16f-6139-4ec2-8042-7cfa04d6a733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:17.348515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-216bc16f-6139-4ec2-8042-7cfa04d6a733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:17.359862 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:17.414566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20a4b8-4f05-439b-9317-47fe50c57d72 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:17.419721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2d177-10ae-4bca-91d0-50d93514ee92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:17.420462 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2d177-10ae-4bca-91d0-50d93514ee92 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:17.420866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2d177-10ae-4bca-91d0-50d93514ee92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:17.421210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2d177-10ae-4bca-91d0-50d93514ee92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:17.421922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2d177-10ae-4bca-91d0-50d93514ee92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:17.424337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2d177-10ae-4bca-91d0-50d93514ee92 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:17.451214 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216bc16f-6139-4ec2-8042-7cfa04d6a733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:17.452838 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216bc16f-6139-4ec2-8042-7cfa04d6a733 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:17.465237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1169/2330] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:17:17 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:17:17.466469 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2d177-10ae-4bca-91d0-50d93514ee92 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:17.467729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1162/2331] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:17:17.544939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2d177-10ae-4bca-91d0-50d93514ee92 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:17.557984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d073e5d-ebd4-47b7-9722-58d4bcde54a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:17.558631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d073e5d-ebd4-47b7-9722-58d4bcde54a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:17.559047 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d073e5d-ebd4-47b7-9722-58d4bcde54a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:17.559601 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d073e5d-ebd4-47b7-9722-58d4bcde54a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:17.560345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d073e5d-ebd4-47b7-9722-58d4bcde54a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:17.562450 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d073e5d-ebd4-47b7-9722-58d4bcde54a5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:17.591576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1170/2332] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:17:17 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:17:17.597312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bc16f-6139-4ec2-8042-7cfa04d6a733 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:17.610335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bc16f-6139-4ec2-8042-7cfa04d6a733 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:17.615934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b6199-595d-4c78-baa8-b10845e88c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:17.616569 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b6199-595d-4c78-baa8-b10845e88c4f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:17.616873 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b6199-595d-4c78-baa8-b10845e88c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:17.617143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b6199-595d-4c78-baa8-b10845e88c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:17.617804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b6199-595d-4c78-baa8-b10845e88c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:17.619473 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b6199-595d-4c78-baa8-b10845e88c4f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:17.633825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1163/2333] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:17:17.634572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d073e5d-ebd4-47b7-9722-58d4bcde54a5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:17.662915 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d073e5d-ebd4-47b7-9722-58d4bcde54a5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:17.669505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91b160-1d28-47cf-bc04-e51ede00b82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:17.670694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91b160-1d28-47cf-bc04-e51ede00b82d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:17.671134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91b160-1d28-47cf-bc04-e51ede00b82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:17.671748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91b160-1d28-47cf-bc04-e51ede00b82d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:17.672255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91b160-1d28-47cf-bc04-e51ede00b82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:17.674413 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91b160-1d28-47cf-bc04-e51ede00b82d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:17.686339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1171/2334] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:17:17.690503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b6199-595d-4c78-baa8-b10845e88c4f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:17.701482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b6199-595d-4c78-baa8-b10845e88c4f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:17.706986 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c7799-b590-4999-a82d-4d91c7fe2b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:17.707707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c7799-b590-4999-a82d-4d91c7fe2b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:17.707977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c7799-b590-4999-a82d-4d91c7fe2b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:17.708246 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c7799-b590-4999-a82d-4d91c7fe2b17 admin admin] PATH_INFO: `/v3/projects/fd90c9c3a5554874b6b1d2e5d32d11f9/users/b9d62ee769bb4097b02703cca87ee6f4/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:17.724885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c7799-b590-4999-a82d-4d91c7fe2b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd90c9c3a5554874b6b1d2e5d32d11f9, user_id=b9d62ee769bb4097b02703cca87ee6f4, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:17.727557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c7799-b590-4999-a82d-4d91c7fe2b17 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:17.751764 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1164/2335] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:17:17 2022] PUT /identity/v3/projects/fd90c9c3a5554874b6b1d2e5d32d11f9/users/b9d62ee769bb4097b02703cca87ee6f4/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:17:17.753625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c602d-c97c-45bb-b9e9-3de6075139a7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:17.754005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c602d-c97c-45bb-b9e9-3de6075139a7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:17.754616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c602d-c97c-45bb-b9e9-3de6075139a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:17.781662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1e3c602d-c97c-45bb-b9e9-3de6075139a7 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:17.782913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1e3c602d-c97c-45bb-b9e9-3de6075139a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:17.807200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c7799-b590-4999-a82d-4d91c7fe2b17 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:17.809948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-436c7799-b590-4999-a82d-4d91c7fe2b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:17.858045 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1172/2336] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:17.869064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c602d-c97c-45bb-b9e9-3de6075139a7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:17.870482 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1e3c602d-c97c-45bb-b9e9-3de6075139a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:17.878123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:17.901143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c7799-b590-4999-a82d-4d91c7fe2b17 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:17.906766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:17.907418 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:17.907694 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:17.907967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] PATH_INFO: `/v3/projects/761ca42525ff4a8f8b8bcaf6ff364320` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:17.909125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=761ca42525ff4a8f8b8bcaf6ff364320)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:17.910174 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:17.911106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:17.938655 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c602d-c97c-45bb-b9e9-3de6075139a7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:17.943342 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27725e0d-adb8-41a0-b8f7-7cff455c7574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:17.944051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27725e0d-adb8-41a0-b8f7-7cff455c7574 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:17.944332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27725e0d-adb8-41a0-b8f7-7cff455c7574 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:17.944593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27725e0d-adb8-41a0-b8f7-7cff455c7574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:17.954094 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27725e0d-adb8-41a0-b8f7-7cff455c7574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:17.955567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27725e0d-adb8-41a0-b8f7-7cff455c7574 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:17.962059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '761ca42525ff4a8f8b8bcaf6ff364320', 'request_id': 'req-51d2ea14-8d34-4304-8f91-a74d26f82174'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:17.965964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1173/2337] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:17:17.973454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '761ca42525ff4a8f8b8bcaf6ff364320', 'request_id': 'req-51d2ea14-8d34-4304-8f91-a74d26f82174'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:17.984007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '761ca42525ff4a8f8b8bcaf6ff364320', 'request_id': 'req-51d2ea14-8d34-4304-8f91-a74d26f82174'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:18.002153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] The token cache is being invalidate because project 761ca42525ff4a8f8b8bcaf6ff364320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:17:18.002497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:18.003216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:18.008266 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1165/2338] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:17:17 2022] DELETE /identity/v3/projects/761ca42525ff4a8f8b8bcaf6ff364320 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:17:18.009218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27725e0d-adb8-41a0-b8f7-7cff455c7574 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:18.022740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27725e0d-adb8-41a0-b8f7-7cff455c7574 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:18.101942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40ac7f-3ae3-4edf-84e5-b42939a08eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:18.102586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ac7f-3ae3-4edf-84e5-b42939a08eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:18.102870 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ac7f-3ae3-4edf-84e5-b42939a08eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:18.103128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ac7f-3ae3-4edf-84e5-b42939a08eb4 admin admin] PATH_INFO: `/v3/users/6af68ddcbf1749e081620b05b7bca02f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:18.104261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40ac7f-3ae3-4edf-84e5-b42939a08eb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6af68ddcbf1749e081620b05b7bca02f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:18.106237 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40ac7f-3ae3-4edf-84e5-b42939a08eb4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:18.195585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ce40ac7f-3ae3-4edf-84e5-b42939a08eb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6af68ddcbf1749e081620b05b7bca02f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:18.196241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ce40ac7f-3ae3-4edf-84e5-b42939a08eb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6af68ddcbf1749e081620b05b7bca02f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:18.203592 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ce40ac7f-3ae3-4edf-84e5-b42939a08eb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6af68ddcbf1749e081620b05b7bca02f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:18.219596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ce40ac7f-3ae3-4edf-84e5-b42939a08eb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6af68ddcbf1749e081620b05b7bca02f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:18.220355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ce40ac7f-3ae3-4edf-84e5-b42939a08eb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6af68ddcbf1749e081620b05b7bca02f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:18.237566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1174/2339] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:17:18 2022] DELETE /identity/v3/users/6af68ddcbf1749e081620b05b7bca02f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:18.579413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:18.587906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:18.615510 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2ea14-8d34-4304-8f91-a74d26f82174 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:18.697910 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b931b12-685f-43c5-96bc-83c1d374910c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:18.698924 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b931b12-685f-43c5-96bc-83c1d374910c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:18.699356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b931b12-685f-43c5-96bc-83c1d374910c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:18.699929 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b931b12-685f-43c5-96bc-83c1d374910c admin admin] PATH_INFO: `/v3/projects/8035e71c1f4f43c28ce77b0bead14ce2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:18.701546 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b931b12-685f-43c5-96bc-83c1d374910c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8035e71c1f4f43c28ce77b0bead14ce2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:18.703847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b931b12-685f-43c5-96bc-83c1d374910c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:18.768255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1b931b12-685f-43c5-96bc-83c1d374910c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8035e71c1f4f43c28ce77b0bead14ce2', 'request_id': 'req-1b931b12-685f-43c5-96bc-83c1d374910c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:18.779200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1b931b12-685f-43c5-96bc-83c1d374910c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8035e71c1f4f43c28ce77b0bead14ce2', 'request_id': 'req-1b931b12-685f-43c5-96bc-83c1d374910c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:18.787244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1b931b12-685f-43c5-96bc-83c1d374910c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8035e71c1f4f43c28ce77b0bead14ce2', 'request_id': 'req-1b931b12-685f-43c5-96bc-83c1d374910c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:18.811256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:18.819288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1b931b12-685f-43c5-96bc-83c1d374910c admin admin] The token cache is being invalidate because project 8035e71c1f4f43c28ce77b0bead14ce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:17:18.819580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1b931b12-685f-43c5-96bc-83c1d374910c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:18.820307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1b931b12-685f-43c5-96bc-83c1d374910c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:18.824067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1166/2340] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:17:18 2022] DELETE /identity/v3/projects/8035e71c1f4f43c28ce77b0bead14ce2 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:18.829525 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ab025-39bc-41b0-9112-2f3674e86589 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:18.829813 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ab025-39bc-41b0-9112-2f3674e86589 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:18.830356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ab025-39bc-41b0-9112-2f3674e86589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:18.865914 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ca5ab025-39bc-41b0-9112-2f3674e86589 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:18.868136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ca5ab025-39bc-41b0-9112-2f3674e86589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:18.936868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:18.963362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1175/2341] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:18.968176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54b545-30f4-4586-a0fa-3fc655284bb2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:18.968609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54b545-30f4-4586-a0fa-3fc655284bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:18.969134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54b545-30f4-4586-a0fa-3fc655284bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:18.996345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-df54b545-30f4-4586-a0fa-3fc655284bb2 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:18.997771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-df54b545-30f4-4586-a0fa-3fc655284bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:19.009888 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1167/2342] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:19.011264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ab025-39bc-41b0-9112-2f3674e86589 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:19.021846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ab025-39bc-41b0-9112-2f3674e86589 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:19.030556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a7c80-1d64-4fe6-9874-502630427704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:19.031199 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a7c80-1d64-4fe6-9874-502630427704 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:19.031481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a7c80-1d64-4fe6-9874-502630427704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:19.031753 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a7c80-1d64-4fe6-9874-502630427704 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:19.032418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a7c80-1d64-4fe6-9874-502630427704 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:17:19.032748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a7c80-1d64-4fe6-9874-502630427704 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:19.034415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a7c80-1d64-4fe6-9874-502630427704 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:19.055361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1176/2343] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:17:19.070747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54b545-30f4-4586-a0fa-3fc655284bb2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:19.092387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54b545-30f4-4586-a0fa-3fc655284bb2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:19.109297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b0fff-56f0-406a-837c-a5f23914399c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:19.109845 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b0fff-56f0-406a-837c-a5f23914399c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:19.110466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b0fff-56f0-406a-837c-a5f23914399c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:19.110466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b0fff-56f0-406a-837c-a5f23914399c admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:19.110989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b0fff-56f0-406a-837c-a5f23914399c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:19.113497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b0fff-56f0-406a-837c-a5f23914399c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:19.159549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1168/2344] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:17:19 2022] POST /identity/v3/projects => generated 381 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:17:19.164451 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a7c80-1d64-4fe6-9874-502630427704 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:19.176066 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a7c80-1d64-4fe6-9874-502630427704 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:19.284471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95877e1f-3673-4640-ad2a-6150b937ba3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:19.285217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95877e1f-3673-4640-ad2a-6150b937ba3d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:19.285438 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95877e1f-3673-4640-ad2a-6150b937ba3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:19.285698 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95877e1f-3673-4640-ad2a-6150b937ba3d admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:19.287109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95877e1f-3673-4640-ad2a-6150b937ba3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:19.289420 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95877e1f-3673-4640-ad2a-6150b937ba3d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:19.327514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1177/2345] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:17:19 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:17:19.328730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b0fff-56f0-406a-837c-a5f23914399c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:19.348411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b0fff-56f0-406a-837c-a5f23914399c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:19.354763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b79fc9-d153-479f-a091-837b71b08ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:19.355411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b79fc9-d153-479f-a091-837b71b08ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:19.356281 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b79fc9-d153-479f-a091-837b71b08ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:19.356281 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b79fc9-d153-479f-a091-837b71b08ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:19.356794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b79fc9-d153-479f-a091-837b71b08ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:19.358501 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b79fc9-d153-479f-a091-837b71b08ab3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:19.372786 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1169/2346] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:17:19.376419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95877e1f-3673-4640-ad2a-6150b937ba3d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:19.388582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95877e1f-3673-4640-ad2a-6150b937ba3d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:19.395274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb9a6f-2991-4c64-a919-6ef2a6724b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:19.395759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb9a6f-2991-4c64-a919-6ef2a6724b42 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:19.396090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb9a6f-2991-4c64-a919-6ef2a6724b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:19.396364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb9a6f-2991-4c64-a919-6ef2a6724b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:19.397051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb9a6f-2991-4c64-a919-6ef2a6724b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:19.398869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb9a6f-2991-4c64-a919-6ef2a6724b42 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:19.434910 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1178/2347] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:17:19.444609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b79fc9-d153-479f-a091-837b71b08ab3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:19.459777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b79fc9-d153-479f-a091-837b71b08ab3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:19.465889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b93dc6-6498-446f-bdaa-18fa856388fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:19.467874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b93dc6-6498-446f-bdaa-18fa856388fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:19.468229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b93dc6-6498-446f-bdaa-18fa856388fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:19.469076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b93dc6-6498-446f-bdaa-18fa856388fe admin admin] PATH_INFO: `/v3/projects/305e564aae3c4cf5988d6cab7eb33935/users/7f3c6fe7f9f64fb89a7918bd641a0844/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:19.491542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b93dc6-6498-446f-bdaa-18fa856388fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=305e564aae3c4cf5988d6cab7eb33935, user_id=7f3c6fe7f9f64fb89a7918bd641a0844, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:19.494501 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b93dc6-6498-446f-bdaa-18fa856388fe admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:19.515896 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1170/2348] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:17:19 2022] PUT /identity/v3/projects/305e564aae3c4cf5988d6cab7eb33935/users/7f3c6fe7f9f64fb89a7918bd641a0844/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:19.529011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673799a-0270-42a6-a13c-e593c809a9b8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:19.529577 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673799a-0270-42a6-a13c-e593c809a9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:19.529852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673799a-0270-42a6-a13c-e593c809a9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:19.565856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8673799a-0270-42a6-a13c-e593c809a9b8 None None] MFA Rules not processed for user `7f3c6fe7f9f64fb89a7918bd641a0844`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:19.567402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8673799a-0270-42a6-a13c-e593c809a9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:19.697424 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1179/2349] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:17:19.701454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b93dc6-6498-446f-bdaa-18fa856388fe admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:19.793523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b93dc6-6498-446f-bdaa-18fa856388fe admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:19.799885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f8824-fe73-43d0-9443-d4140617e2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:19.800496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f8824-fe73-43d0-9443-d4140617e2eb admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:19.800805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f8824-fe73-43d0-9443-d4140617e2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:19.801073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f8824-fe73-43d0-9443-d4140617e2eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:19.801722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f8824-fe73-43d0-9443-d4140617e2eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:19.803710 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f8824-fe73-43d0-9443-d4140617e2eb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:19.840801 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1171/2350] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:17:19 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:17:19.845034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673799a-0270-42a6-a13c-e593c809a9b8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:19.859428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673799a-0270-42a6-a13c-e593c809a9b8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:19.923046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:19.943185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a2f0c-e4b7-44e6-9051-7a3a471686f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:19.943347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a2f0c-e4b7-44e6-9051-7a3a471686f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:19.943619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a2f0c-e4b7-44e6-9051-7a3a471686f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:19.943994 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a2f0c-e4b7-44e6-9051-7a3a471686f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:19.945215 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a2f0c-e4b7-44e6-9051-7a3a471686f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:19.947310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a2f0c-e4b7-44e6-9051-7a3a471686f2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:19.975854 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1180/2351] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:17:19 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:17:19.979974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f8824-fe73-43d0-9443-d4140617e2eb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:19.994095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f8824-fe73-43d0-9443-d4140617e2eb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:20.006158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789f20d-a832-4f74-b1d2-36760b8f27cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:20.006800 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789f20d-a832-4f74-b1d2-36760b8f27cb admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:20.007079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789f20d-a832-4f74-b1d2-36760b8f27cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:20.007349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789f20d-a832-4f74-b1d2-36760b8f27cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:20.008012 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789f20d-a832-4f74-b1d2-36760b8f27cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:20.009666 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789f20d-a832-4f74-b1d2-36760b8f27cb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:20.024315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1172/2352] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:17:20.026624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a2f0c-e4b7-44e6-9051-7a3a471686f2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:20.037972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a2f0c-e4b7-44e6-9051-7a3a471686f2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:20.044176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c3701-5e1f-419d-8fa6-b9804c485cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:20.045497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c3701-5e1f-419d-8fa6-b9804c485cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:20.045933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c3701-5e1f-419d-8fa6-b9804c485cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:20.046210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c3701-5e1f-419d-8fa6-b9804c485cdd admin admin] PATH_INFO: `/v3/projects/79e157cbceb449cb944096e83212be77/users/c0c61c8b995749bb89dbe29ebbd856b3/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:20.067259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c3701-5e1f-419d-8fa6-b9804c485cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e157cbceb449cb944096e83212be77, user_id=c0c61c8b995749bb89dbe29ebbd856b3, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:20.071966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c3701-5e1f-419d-8fa6-b9804c485cdd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:20.090061 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1181/2353] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:17:20 2022] PUT /identity/v3/projects/79e157cbceb449cb944096e83212be77/users/c0c61c8b995749bb89dbe29ebbd856b3/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:20.093872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789f20d-a832-4f74-b1d2-36760b8f27cb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:20.104897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789f20d-a832-4f74-b1d2-36760b8f27cb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:20.197347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebcf65-6695-458c-b003-27f1359b6fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:20.197970 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebcf65-6695-458c-b003-27f1359b6fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:20.198248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebcf65-6695-458c-b003-27f1359b6fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:20.198513 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebcf65-6695-458c-b003-27f1359b6fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:20.199333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebcf65-6695-458c-b003-27f1359b6fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:20.201388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebcf65-6695-458c-b003-27f1359b6fb1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:20.217848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1173/2354] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:17:20.230096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c3701-5e1f-419d-8fa6-b9804c485cdd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:20.242345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c3701-5e1f-419d-8fa6-b9804c485cdd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:20.249776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20989e-9f50-4db4-8e37-3d02925c9515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:20.250594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20989e-9f50-4db4-8e37-3d02925c9515 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:20.250881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20989e-9f50-4db4-8e37-3d02925c9515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:20.251157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20989e-9f50-4db4-8e37-3d02925c9515 admin admin] PATH_INFO: `/v3/projects/79e157cbceb449cb944096e83212be77/users/c0c61c8b995749bb89dbe29ebbd856b3/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:20.253480 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20989e-9f50-4db4-8e37-3d02925c9515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e157cbceb449cb944096e83212be77, user_id=c0c61c8b995749bb89dbe29ebbd856b3, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:20.256333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20989e-9f50-4db4-8e37-3d02925c9515 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:20.269862 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1182/2355] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:17:20 2022] PUT /identity/v3/projects/79e157cbceb449cb944096e83212be77/users/c0c61c8b995749bb89dbe29ebbd856b3/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:20.282087 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9951-73d5-4c33-9057-c3c5f3885ca5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:20.282616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9951-73d5-4c33-9057-c3c5f3885ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:20.283314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9951-73d5-4c33-9057-c3c5f3885ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:20.300350 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:20.333988 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0f9e9951-73d5-4c33-9057-c3c5f3885ca5 None None] MFA Rules not processed for user `c0c61c8b995749bb89dbe29ebbd856b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:20.335236 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0f9e9951-73d5-4c33-9057-c3c5f3885ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:20.430491 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1174/2356] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:20.442376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20989e-9f50-4db4-8e37-3d02925c9515 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:20.444436 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-df20989e-9f50-4db4-8e37-3d02925c9515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:20.579102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20989e-9f50-4db4-8e37-3d02925c9515 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:20.589463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42daf137-1f0f-4848-9601-35eec63289e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:20.590108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42daf137-1f0f-4848-9601-35eec63289e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:20.590390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42daf137-1f0f-4848-9601-35eec63289e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:20.590666 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42daf137-1f0f-4848-9601-35eec63289e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:20.615233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42daf137-1f0f-4848-9601-35eec63289e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:20.616805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42daf137-1f0f-4848-9601-35eec63289e5 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:20.639019 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1183/2357] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:17:20.756623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e9951-73d5-4c33-9057-c3c5f3885ca5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:20.775538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e9951-73d5-4c33-9057-c3c5f3885ca5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:20.779620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59825f6-766e-4028-9fc5-ac8330c4d03f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:20.780388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59825f6-766e-4028-9fc5-ac8330c4d03f service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:20.780811 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59825f6-766e-4028-9fc5-ac8330c4d03f service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:20.781179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59825f6-766e-4028-9fc5-ac8330c4d03f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:20.887541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59825f6-766e-4028-9fc5-ac8330c4d03f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:20.890375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59825f6-766e-4028-9fc5-ac8330c4d03f service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:20.907477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1175/2358] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:17:21.352415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42daf137-1f0f-4848-9601-35eec63289e5 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:21.493791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42daf137-1f0f-4848-9601-35eec63289e5 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:21.500523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:21.501566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:21.502194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:21.502638 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] PATH_INFO: `/v3/users/7f3c6fe7f9f64fb89a7918bd641a0844` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:21.504255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f3c6fe7f9f64fb89a7918bd641a0844)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:21.507176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:21.574749 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:21.588355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f3c6fe7f9f64fb89a7918bd641a0844'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:21.589766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f3c6fe7f9f64fb89a7918bd641a0844'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:21.610898 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f3c6fe7f9f64fb89a7918bd641a0844'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:21.635550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f3c6fe7f9f64fb89a7918bd641a0844'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:21.636734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f3c6fe7f9f64fb89a7918bd641a0844'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:21.673230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1184/2359] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:17:21 2022] DELETE /identity/v3/users/7f3c6fe7f9f64fb89a7918bd641a0844 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:21.683825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a9d86-b642-48b1-a38a-29382f53f6a7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:21.684110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a9d86-b642-48b1-a38a-29382f53f6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:21.684466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a9d86-b642-48b1-a38a-29382f53f6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:21.710271 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e46a9d86-b642-48b1-a38a-29382f53f6a7 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:21.711516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e46a9d86-b642-48b1-a38a-29382f53f6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:21.826458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1176/2360] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:17:21.836459 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:21.839756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:21.926780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c9195-0751-4fd4-a9dd-edab9f5cbc4a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:21.931483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199662f8-fa0b-46e3-9dd5-c9e02752a435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:21.932177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199662f8-fa0b-46e3-9dd5-c9e02752a435 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:21.932522 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199662f8-fa0b-46e3-9dd5-c9e02752a435 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:21.933356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199662f8-fa0b-46e3-9dd5-c9e02752a435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:21.944025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199662f8-fa0b-46e3-9dd5-c9e02752a435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:21.950797 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199662f8-fa0b-46e3-9dd5-c9e02752a435 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:21.963626 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1185/2361] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:17:22.514477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a9d86-b642-48b1-a38a-29382f53f6a7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:22.516697 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e46a9d86-b642-48b1-a38a-29382f53f6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:22.532148 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a9d86-b642-48b1-a38a-29382f53f6a7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:22.537129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f68a4c-beff-4452-95ce-f5e00558db60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:22.537894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f68a4c-beff-4452-95ce-f5e00558db60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:22.538183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f68a4c-beff-4452-95ce-f5e00558db60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:22.538444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f68a4c-beff-4452-95ce-f5e00558db60 admin admin] PATH_INFO: `/v3/projects/305e564aae3c4cf5988d6cab7eb33935` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:22.539565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f68a4c-beff-4452-95ce-f5e00558db60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=305e564aae3c4cf5988d6cab7eb33935)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:22.541600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f68a4c-beff-4452-95ce-f5e00558db60 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:22.577812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:22.607333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f5f68a4c-beff-4452-95ce-f5e00558db60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '305e564aae3c4cf5988d6cab7eb33935', 'request_id': 'req-f5f68a4c-beff-4452-95ce-f5e00558db60'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:22.621795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f5f68a4c-beff-4452-95ce-f5e00558db60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '305e564aae3c4cf5988d6cab7eb33935', 'request_id': 'req-f5f68a4c-beff-4452-95ce-f5e00558db60'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:22.634226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f5f68a4c-beff-4452-95ce-f5e00558db60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '305e564aae3c4cf5988d6cab7eb33935', 'request_id': 'req-f5f68a4c-beff-4452-95ce-f5e00558db60'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:22.651453 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f5f68a4c-beff-4452-95ce-f5e00558db60 admin admin] The token cache is being invalidate because project 305e564aae3c4cf5988d6cab7eb33935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:17:22.651998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f5f68a4c-beff-4452-95ce-f5e00558db60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:22.653078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f5f68a4c-beff-4452-95ce-f5e00558db60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:22.655478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1177/2362] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:17:22 2022] DELETE /identity/v3/projects/305e564aae3c4cf5988d6cab7eb33935 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:22.660195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199662f8-fa0b-46e3-9dd5-c9e02752a435 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:22.675613 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199662f8-fa0b-46e3-9dd5-c9e02752a435 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:22.766967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:22.767952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:22.768301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:22.768636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] PATH_INFO: `/v3/users/c0c61c8b995749bb89dbe29ebbd856b3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:22.769883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0c61c8b995749bb89dbe29ebbd856b3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:22.771945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:22.831615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0c61c8b995749bb89dbe29ebbd856b3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:22.832289 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0c61c8b995749bb89dbe29ebbd856b3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:22.846376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0c61c8b995749bb89dbe29ebbd856b3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:22.857195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:22.866534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0c61c8b995749bb89dbe29ebbd856b3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:22.867860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0c61c8b995749bb89dbe29ebbd856b3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:22.896378 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1186/2363] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:17:22 2022] DELETE /identity/v3/users/c0c61c8b995749bb89dbe29ebbd856b3 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:17:23.174351 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f5b5b-c0fd-4557-b0e1-effdd67ddcf8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:23.174819 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f5b5b-c0fd-4557-b0e1-effdd67ddcf8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:23.175085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f5b5b-c0fd-4557-b0e1-effdd67ddcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:23.215379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-804f5b5b-c0fd-4557-b0e1-effdd67ddcf8 None None] MFA Rules not processed for user `b9d62ee769bb4097b02703cca87ee6f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:23.216526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-804f5b5b-c0fd-4557-b0e1-effdd67ddcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:23.275820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:23.277597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:23.345631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1178/2364] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:23.350372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f5b5b-c0fd-4557-b0e1-effdd67ddcf8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:23.351675 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-804f5b5b-c0fd-4557-b0e1-effdd67ddcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:23.426549 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e2852-154c-4a92-b56b-8791a085c348 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:23.455622 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcf140-0270-4224-8cc9-0fbd671393a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:23.460781 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcf140-0270-4224-8cc9-0fbd671393a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:23.462661 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcf140-0270-4224-8cc9-0fbd671393a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:23.465380 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcf140-0270-4224-8cc9-0fbd671393a9 admin admin] PATH_INFO: `/v3/projects/79e157cbceb449cb944096e83212be77` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:23.470528 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebcf140-0270-4224-8cc9-0fbd671393a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e157cbceb449cb944096e83212be77)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:23.478841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebcf140-0270-4224-8cc9-0fbd671393a9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:23.490385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f5b5b-c0fd-4557-b0e1-effdd67ddcf8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:23.498647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061bfac-6e1a-41a5-810e-5ff4d00be05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:23.499515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061bfac-6e1a-41a5-810e-5ff4d00be05f admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:23.499702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061bfac-6e1a-41a5-810e-5ff4d00be05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:23.499973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061bfac-6e1a-41a5-810e-5ff4d00be05f admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:23.500634 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061bfac-6e1a-41a5-810e-5ff4d00be05f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:23.503158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061bfac-6e1a-41a5-810e-5ff4d00be05f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:23.562617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1179/2365] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:17:23 2022] POST /identity/v3/projects => generated 371 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:17:23.563982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061bfac-6e1a-41a5-810e-5ff4d00be05f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:23.570193 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5ebcf140-0270-4224-8cc9-0fbd671393a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e157cbceb449cb944096e83212be77', 'request_id': 'req-5ebcf140-0270-4224-8cc9-0fbd671393a9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:23.578149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5ebcf140-0270-4224-8cc9-0fbd671393a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e157cbceb449cb944096e83212be77', 'request_id': 'req-5ebcf140-0270-4224-8cc9-0fbd671393a9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:23.585536 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5ebcf140-0270-4224-8cc9-0fbd671393a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e157cbceb449cb944096e83212be77', 'request_id': 'req-5ebcf140-0270-4224-8cc9-0fbd671393a9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:23.617205 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:23.619259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5ebcf140-0270-4224-8cc9-0fbd671393a9 admin admin] The token cache is being invalidate because project 79e157cbceb449cb944096e83212be77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:17:23.620120 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5ebcf140-0270-4224-8cc9-0fbd671393a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:23.622819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5ebcf140-0270-4224-8cc9-0fbd671393a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:17:23.625628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1187/2366] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:17:23 2022] DELETE /identity/v3/projects/79e157cbceb449cb944096e83212be77 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:23.635483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a507f11-583e-4928-9c31-d4232a3cc619 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:23.635917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a507f11-583e-4928-9c31-d4232a3cc619 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:23.636708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a507f11-583e-4928-9c31-d4232a3cc619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:23.667507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7a507f11-583e-4928-9c31-d4232a3cc619 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:23.669012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7a507f11-583e-4928-9c31-d4232a3cc619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:23.678010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061bfac-6e1a-41a5-810e-5ff4d00be05f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:23.679613 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1188/2367] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:23.684996 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38bd92-4a65-4a76-a1bf-c4dc276725ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:23.685758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38bd92-4a65-4a76-a1bf-c4dc276725ca admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:23.686038 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38bd92-4a65-4a76-a1bf-c4dc276725ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:23.686307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38bd92-4a65-4a76-a1bf-c4dc276725ca admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:23.686990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f38bd92-4a65-4a76-a1bf-c4dc276725ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:23.688546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e410b1-5240-4345-b25c-aa092560a529 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:23.689031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e410b1-5240-4345-b25c-aa092560a529 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:23.689363 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e410b1-5240-4345-b25c-aa092560a529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:23.695302 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f38bd92-4a65-4a76-a1bf-c4dc276725ca admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:23.715093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d6e410b1-5240-4345-b25c-aa092560a529 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:23.717080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d6e410b1-5240-4345-b25c-aa092560a529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:23.727624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1189/2368] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:17:23.731822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e410b1-5240-4345-b25c-aa092560a529 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:23.736239 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1180/2369] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:17:23 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:17:23.737325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38bd92-4a65-4a76-a1bf-c4dc276725ca admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:23.739235 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3f38bd92-4a65-4a76-a1bf-c4dc276725ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:23.751456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e410b1-5240-4345-b25c-aa092560a529 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:23.753767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38bd92-4a65-4a76-a1bf-c4dc276725ca admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:23.758003 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa677e-8cae-486f-bf92-4dfc9757e9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:23.758899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29294a8-3fe4-463b-bd9b-aa52f00700a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:23.759032 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa677e-8cae-486f-bf92-4dfc9757e9fa admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:23.759312 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa677e-8cae-486f-bf92-4dfc9757e9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:23.759598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa677e-8cae-486f-bf92-4dfc9757e9fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:23.759770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29294a8-3fe4-463b-bd9b-aa52f00700a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:23.760021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29294a8-3fe4-463b-bd9b-aa52f00700a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:23.760294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29294a8-3fe4-463b-bd9b-aa52f00700a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:23.760529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa677e-8cae-486f-bf92-4dfc9757e9fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:17:23.760860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa677e-8cae-486f-bf92-4dfc9757e9fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:23.761055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29294a8-3fe4-463b-bd9b-aa52f00700a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:23.762835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29294a8-3fe4-463b-bd9b-aa52f00700a4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:23.763071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa677e-8cae-486f-bf92-4dfc9757e9fa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:23.775229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1181/2370] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:17:23.781241 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29294a8-3fe4-463b-bd9b-aa52f00700a4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:23.794523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29294a8-3fe4-463b-bd9b-aa52f00700a4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:23.799494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1190/2371] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:17:23.803973 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa677e-8cae-486f-bf92-4dfc9757e9fa admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:23.805575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001eb85-8d88-4351-8714-147729836312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:23.806259 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001eb85-8d88-4351-8714-147729836312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:23.811273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001eb85-8d88-4351-8714-147729836312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:23.811737 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001eb85-8d88-4351-8714-147729836312 admin admin] PATH_INFO: `/v3/projects/2bb19cbf237f4cd7bb3fb6ea582cc1bb/users/d90fa5e88bd543a68cd5d3c70b0b6b2c/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:23.831683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa677e-8cae-486f-bf92-4dfc9757e9fa admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:23.839065 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff865d-3498-4111-a791-a18b7ac10c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:23.840214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff865d-3498-4111-a791-a18b7ac10c77 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:23.841061 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff865d-3498-4111-a791-a18b7ac10c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:23.841760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff865d-3498-4111-a791-a18b7ac10c77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:23.842705 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff865d-3498-4111-a791-a18b7ac10c77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:23.846011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff865d-3498-4111-a791-a18b7ac10c77 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:23.855498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001eb85-8d88-4351-8714-147729836312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb19cbf237f4cd7bb3fb6ea582cc1bb, user_id=d90fa5e88bd543a68cd5d3c70b0b6b2c, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:23.862833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001eb85-8d88-4351-8714-147729836312 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:23.899016 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1182/2372] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:17:23 2022] PUT /identity/v3/projects/2bb19cbf237f4cd7bb3fb6ea582cc1bb/users/d90fa5e88bd543a68cd5d3c70b0b6b2c/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:23.906576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001eb85-8d88-4351-8714-147729836312 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:23.914895 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1191/2373] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:17:23 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:17:23.919072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff865d-3498-4111-a791-a18b7ac10c77 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:23.981271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:24.039693 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff865d-3498-4111-a791-a18b7ac10c77 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:24.045320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c05c41-28e0-430a-bb97-45b0f2e6a7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:24.046350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c05c41-28e0-430a-bb97-45b0f2e6a7d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:24.047390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c05c41-28e0-430a-bb97-45b0f2e6a7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:24.048957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c05c41-28e0-430a-bb97-45b0f2e6a7d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:24.050132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c05c41-28e0-430a-bb97-45b0f2e6a7d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:24.054013 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c05c41-28e0-430a-bb97-45b0f2e6a7d2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:24.062356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001eb85-8d88-4351-8714-147729836312 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:24.068533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d60f2d-756b-49c4-9d57-d906982455d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:24.076040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d60f2d-756b-49c4-9d57-d906982455d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:24.076333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d60f2d-756b-49c4-9d57-d906982455d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:24.077367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d60f2d-756b-49c4-9d57-d906982455d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:24.079480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d60f2d-756b-49c4-9d57-d906982455d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:24.085633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d60f2d-756b-49c4-9d57-d906982455d2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:24.091008 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1192/2374] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:17:23 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:17:24.097125 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c05c41-28e0-430a-bb97-45b0f2e6a7d2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:24.112329 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1183/2375] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:17:24.112504 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c05c41-28e0-430a-bb97-45b0f2e6a7d2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:24.113792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d60f2d-756b-49c4-9d57-d906982455d2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:24.117729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054c7cc-e06d-4d51-8a7b-f0e3782c6eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:24.118649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054c7cc-e06d-4d51-8a7b-f0e3782c6eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:24.119304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054c7cc-e06d-4d51-8a7b-f0e3782c6eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:24.119670 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054c7cc-e06d-4d51-8a7b-f0e3782c6eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:24.120410 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054c7cc-e06d-4d51-8a7b-f0e3782c6eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:24.122790 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054c7cc-e06d-4d51-8a7b-f0e3782c6eb4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:24.136605 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d60f2d-756b-49c4-9d57-d906982455d2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:24.137728 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1193/2376] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:17:24.149721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054c7cc-e06d-4d51-8a7b-f0e3782c6eb4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:24.158840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a5152-9fcf-4041-b95f-11bf38e68512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:24.159556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a5152-9fcf-4041-b95f-11bf38e68512 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:24.159833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a5152-9fcf-4041-b95f-11bf38e68512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:24.160890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a5152-9fcf-4041-b95f-11bf38e68512 admin admin] PATH_INFO: `/v3/projects/2bb19cbf237f4cd7bb3fb6ea582cc1bb/users/d90fa5e88bd543a68cd5d3c70b0b6b2c/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:24.165405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a5152-9fcf-4041-b95f-11bf38e68512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb19cbf237f4cd7bb3fb6ea582cc1bb, user_id=d90fa5e88bd543a68cd5d3c70b0b6b2c, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:24.168522 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a5152-9fcf-4041-b95f-11bf38e68512 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:24.172054 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054c7cc-e06d-4d51-8a7b-f0e3782c6eb4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:24.177451 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35857396-f1c4-4809-8205-1fbdb5faa168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:24.182488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35857396-f1c4-4809-8205-1fbdb5faa168 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:24.182857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35857396-f1c4-4809-8205-1fbdb5faa168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:24.183137 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35857396-f1c4-4809-8205-1fbdb5faa168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:24.183871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35857396-f1c4-4809-8205-1fbdb5faa168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:24.186167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35857396-f1c4-4809-8205-1fbdb5faa168 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:24.187832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1184/2377] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:17:24 2022] PUT /identity/v3/projects/2bb19cbf237f4cd7bb3fb6ea582cc1bb/users/d90fa5e88bd543a68cd5d3c70b0b6b2c/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:24.204694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1194/2378] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:17:24.206625 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a5152-9fcf-4041-b95f-11bf38e68512 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:24.347250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a5152-9fcf-4041-b95f-11bf38e68512 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:24.357321 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2eea1-e81b-4239-bece-298838552471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:24.358469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2eea1-e81b-4239-bece-298838552471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:24.359740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2eea1-e81b-4239-bece-298838552471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:24.359930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2eea1-e81b-4239-bece-298838552471 admin admin] PATH_INFO: `/v3/projects/de51bce19c0d490ab0c11d4afe5e01fe/users/6582a833830040b3be93cde90b4f489c/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:24.374920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:24.380318 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2eea1-e81b-4239-bece-298838552471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de51bce19c0d490ab0c11d4afe5e01fe, user_id=6582a833830040b3be93cde90b4f489c, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:24.383664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2eea1-e81b-4239-bece-298838552471 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:24.409347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1185/2379] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:17:24 2022] PUT /identity/v3/projects/de51bce19c0d490ab0c11d4afe5e01fe/users/6582a833830040b3be93cde90b4f489c/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:24.415274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89bb37-a1d5-4161-9843-c774115aefe2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:24.415553 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89bb37-a1d5-4161-9843-c774115aefe2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:24.415949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89bb37-a1d5-4161-9843-c774115aefe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:24.445384 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3d89bb37-a1d5-4161-9843-c774115aefe2 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:24.449097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3d89bb37-a1d5-4161-9843-c774115aefe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:24.603876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1195/2380] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:24.609909 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2eea1-e81b-4239-bece-298838552471 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:24.611979 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-53f2eea1-e81b-4239-bece-298838552471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:24.700758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2eea1-e81b-4239-bece-298838552471 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:24.704926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938420c2-0cdd-4381-840c-8c1adf498897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:24.705638 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938420c2-0cdd-4381-840c-8c1adf498897 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:24.705998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938420c2-0cdd-4381-840c-8c1adf498897 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:24.706317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938420c2-0cdd-4381-840c-8c1adf498897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:24.723757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938420c2-0cdd-4381-840c-8c1adf498897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:24.725753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938420c2-0cdd-4381-840c-8c1adf498897 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:24.737580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1186/2381] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:17:25.849672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e844353-fbc0-4770-affd-c4ccc6aaefda None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:25.850200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e844353-fbc0-4770-affd-c4ccc6aaefda None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:25.850200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e844353-fbc0-4770-affd-c4ccc6aaefda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:25.892863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0e844353-fbc0-4770-affd-c4ccc6aaefda None None] MFA Rules not processed for user `6582a833830040b3be93cde90b4f489c`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:25.898696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0e844353-fbc0-4770-affd-c4ccc6aaefda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:26.028289 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1196/2382] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:26.033278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938420c2-0cdd-4381-840c-8c1adf498897 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:26.045759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938420c2-0cdd-4381-840c-8c1adf498897 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:26.052193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc2d7b-7b7a-49c8-8470-e1d298828801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:26.052979 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc2d7b-7b7a-49c8-8470-e1d298828801 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:26.054866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc2d7b-7b7a-49c8-8470-e1d298828801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:26.055369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc2d7b-7b7a-49c8-8470-e1d298828801 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:26.056119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc2d7b-7b7a-49c8-8470-e1d298828801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:26.058202 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc2d7b-7b7a-49c8-8470-e1d298828801 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:26.105967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1187/2383] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:17:26 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:17:26.109671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e844353-fbc0-4770-affd-c4ccc6aaefda None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:26.131995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e844353-fbc0-4770-affd-c4ccc6aaefda None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:26.196593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:26.280988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02487d-0627-42a6-9af3-e23ca443d822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:26.282266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02487d-0627-42a6-9af3-e23ca443d822 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:26.282792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02487d-0627-42a6-9af3-e23ca443d822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:26.283336 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02487d-0627-42a6-9af3-e23ca443d822 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:26.284684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02487d-0627-42a6-9af3-e23ca443d822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:26.286995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02487d-0627-42a6-9af3-e23ca443d822 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:26.348434 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1197/2384] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:17:26 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:17:26.356241 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc2d7b-7b7a-49c8-8470-e1d298828801 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:26.381607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc2d7b-7b7a-49c8-8470-e1d298828801 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:26.397415 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda99e-7f53-4214-bc73-8e67952cd95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:26.398920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda99e-7f53-4214-bc73-8e67952cd95d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:26.399306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda99e-7f53-4214-bc73-8e67952cd95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:26.400918 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda99e-7f53-4214-bc73-8e67952cd95d admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:26.401733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eda99e-7f53-4214-bc73-8e67952cd95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:26.407852 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eda99e-7f53-4214-bc73-8e67952cd95d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:26.438435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1188/2385] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:17:26.442141 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02487d-0627-42a6-9af3-e23ca443d822 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:26.484916 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02487d-0627-42a6-9af3-e23ca443d822 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:26.507866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb12b5-cecd-470a-9eaf-881b2a379637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:26.508710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb12b5-cecd-470a-9eaf-881b2a379637 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:26.509025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb12b5-cecd-470a-9eaf-881b2a379637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:26.509275 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb12b5-cecd-470a-9eaf-881b2a379637 admin admin] PATH_INFO: `/v3/projects/dd54e6d8cda24bf9ae8874d7be98e92d/users/483a1f4cd72047ca86ad0396d5cbc60c/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:26.550001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb12b5-cecd-470a-9eaf-881b2a379637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd54e6d8cda24bf9ae8874d7be98e92d, user_id=483a1f4cd72047ca86ad0396d5cbc60c, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:26.552954 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb12b5-cecd-470a-9eaf-881b2a379637 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:26.593758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1198/2386] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:17:26 2022] PUT /identity/v3/projects/dd54e6d8cda24bf9ae8874d7be98e92d/users/483a1f4cd72047ca86ad0396d5cbc60c/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:26.597815 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda99e-7f53-4214-bc73-8e67952cd95d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:26.617119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda99e-7f53-4214-bc73-8e67952cd95d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:26.750499 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed46e8d-bfd6-4a1a-b7fd-23fb5b9e4ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:26.752379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed46e8d-bfd6-4a1a-b7fd-23fb5b9e4ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:26.752730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed46e8d-bfd6-4a1a-b7fd-23fb5b9e4ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:26.753011 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed46e8d-bfd6-4a1a-b7fd-23fb5b9e4ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:26.754617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed46e8d-bfd6-4a1a-b7fd-23fb5b9e4ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:26.759731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed46e8d-bfd6-4a1a-b7fd-23fb5b9e4ff9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:26.784025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1189/2387] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:17:26.789167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb12b5-cecd-470a-9eaf-881b2a379637 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:26.811052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb12b5-cecd-470a-9eaf-881b2a379637 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:26.851460 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52758ece-95b6-4359-86e6-d1f1df62807c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:26.853933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52758ece-95b6-4359-86e6-d1f1df62807c admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:26.864032 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52758ece-95b6-4359-86e6-d1f1df62807c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:26.864884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52758ece-95b6-4359-86e6-d1f1df62807c admin admin] PATH_INFO: `/v3/projects/dd54e6d8cda24bf9ae8874d7be98e92d/users/483a1f4cd72047ca86ad0396d5cbc60c/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:26.869699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52758ece-95b6-4359-86e6-d1f1df62807c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd54e6d8cda24bf9ae8874d7be98e92d, user_id=483a1f4cd72047ca86ad0396d5cbc60c, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:26.876346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52758ece-95b6-4359-86e6-d1f1df62807c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:26.899757 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1199/2388] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:17:26 2022] PUT /identity/v3/projects/dd54e6d8cda24bf9ae8874d7be98e92d/users/483a1f4cd72047ca86ad0396d5cbc60c/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:17:28.120479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214e6b8-c14a-4e08-b8a2-5de4fa49ad16 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:28.121495 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214e6b8-c14a-4e08-b8a2-5de4fa49ad16 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:28.121911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214e6b8-c14a-4e08-b8a2-5de4fa49ad16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:28.190753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5214e6b8-c14a-4e08-b8a2-5de4fa49ad16 None None] MFA Rules not processed for user `483a1f4cd72047ca86ad0396d5cbc60c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:28.192208 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5214e6b8-c14a-4e08-b8a2-5de4fa49ad16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:28.274697 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:28.356679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1190/2389] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:28.363378 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52758ece-95b6-4359-86e6-d1f1df62807c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:28.365359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-52758ece-95b6-4359-86e6-d1f1df62807c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:28.502140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52758ece-95b6-4359-86e6-d1f1df62807c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:28.518750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325fa31-cbde-4bb5-b184-4042d8b36e77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:28.519695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325fa31-cbde-4bb5-b184-4042d8b36e77 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:28.519994 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325fa31-cbde-4bb5-b184-4042d8b36e77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:28.520266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325fa31-cbde-4bb5-b184-4042d8b36e77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:28.667768 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325fa31-cbde-4bb5-b184-4042d8b36e77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:28.670556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325fa31-cbde-4bb5-b184-4042d8b36e77 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:28.706738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1200/2390] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:17:31.248571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590018d-f7b3-4716-889b-e297c0329378 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:31.249652 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590018d-f7b3-4716-889b-e297c0329378 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:31.250134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590018d-f7b3-4716-889b-e297c0329378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:31.295830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3590018d-f7b3-4716-889b-e297c0329378 None None] MFA Rules not processed for user `d90fa5e88bd543a68cd5d3c70b0b6b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:31.297392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3590018d-f7b3-4716-889b-e297c0329378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:31.388273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1191/2391] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:31.403439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325fa31-cbde-4bb5-b184-4042d8b36e77 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:31.405753 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0325fa31-cbde-4bb5-b184-4042d8b36e77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:31.444582 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:31.497007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325fa31-cbde-4bb5-b184-4042d8b36e77 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:31.507134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61959c00-5e74-4648-989a-4356149f1a3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:31.507618 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61959c00-5e74-4648-989a-4356149f1a3e service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:31.507903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61959c00-5e74-4648-989a-4356149f1a3e service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:31.508313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61959c00-5e74-4648-989a-4356149f1a3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:31.510282 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-61959c00-5e74-4648-989a-4356149f1a3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:31.558474 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590018d-f7b3-4716-889b-e297c0329378 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:17:31.572211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590018d-f7b3-4716-889b-e297c0329378 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:17:31.578185 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de6e88-fbce-4d94-9d00-42a89f969068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:17:31.579677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de6e88-fbce-4d94-9d00-42a89f969068 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:31.580352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de6e88-fbce-4d94-9d00-42a89f969068 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:31.580816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de6e88-fbce-4d94-9d00-42a89f969068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:31.598239 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de6e88-fbce-4d94-9d00-42a89f969068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:31.600271 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de6e88-fbce-4d94-9d00-42a89f969068 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:31.606633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61959c00-5e74-4648-989a-4356149f1a3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:17:31.608826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61959c00-5e74-4648-989a-4356149f1a3e service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:17:31.622084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1201/2392] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:17:31.632750 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1192/2393] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:17:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:17:32.822552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0737f-30f8-4f37-b5bf-89bb0ef874f7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:32.823186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0737f-30f8-4f37-b5bf-89bb0ef874f7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:32.823186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0737f-30f8-4f37-b5bf-89bb0ef874f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:32.844761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c4b0737f-30f8-4f37-b5bf-89bb0ef874f7 None None] MFA Rules not processed for user `b9d62ee769bb4097b02703cca87ee6f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:32.846046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c4b0737f-30f8-4f37-b5bf-89bb0ef874f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:32.854575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1202/2394] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:17:41.609843 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=49 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:42.838088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=45 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:17:56.750588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73f90c-c591-4209-a102-1df428dd9416 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:17:56.751107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73f90c-c591-4209-a102-1df428dd9416 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:17:56.751107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73f90c-c591-4209-a102-1df428dd9416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:17:56.771232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2e73f90c-c591-4209-a102-1df428dd9416 None None] MFA Rules not processed for user `6d0c1c510b0a4fa39927f7f92352d034`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:17:56.772713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2e73f90c-c591-4209-a102-1df428dd9416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:17:56.829580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1193/2395] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:17:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:18:06.822587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=25 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:09.781540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0737f-30f8-4f37-b5bf-89bb0ef874f7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:09.851353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0737f-30f8-4f37-b5bf-89bb0ef874f7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:09.856175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:09.856818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:09.857116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:09.857405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] PATH_INFO: `/v3/users/6582a833830040b3be93cde90b4f489c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:09.858518 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6582a833830040b3be93cde90b4f489c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:09.860452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:09.914456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6582a833830040b3be93cde90b4f489c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:09.915128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6582a833830040b3be93cde90b4f489c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:09.923715 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6582a833830040b3be93cde90b4f489c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:09.938306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6582a833830040b3be93cde90b4f489c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:09.939101 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6582a833830040b3be93cde90b4f489c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:09.953578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1203/2396] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:18:09 2022] DELETE /identity/v3/users/6582a833830040b3be93cde90b4f489c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:18:10.166557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73f90c-c591-4209-a102-1df428dd9416 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:10.189440 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73f90c-c591-4209-a102-1df428dd9416 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:10.257700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:10.258496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:10.258775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:10.259040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] PATH_INFO: `/v3/projects/de51bce19c0d490ab0c11d4afe5e01fe` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:10.260213 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de51bce19c0d490ab0c11d4afe5e01fe)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:10.262190 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:10.312170 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de51bce19c0d490ab0c11d4afe5e01fe', 'request_id': 'req-3ba7b920-5427-4738-9da9-e384bfe8ac07'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.326576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de51bce19c0d490ab0c11d4afe5e01fe', 'request_id': 'req-3ba7b920-5427-4738-9da9-e384bfe8ac07'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.333712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de51bce19c0d490ab0c11d4afe5e01fe', 'request_id': 'req-3ba7b920-5427-4738-9da9-e384bfe8ac07'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.344828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] The token cache is being invalidate because project de51bce19c0d490ab0c11d4afe5e01fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:18:10.345114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.345776 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.347435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1194/2397] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:18:10 2022] DELETE /identity/v3/projects/de51bce19c0d490ab0c11d4afe5e01fe => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:18:10.351479 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:10.353073 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:10.368276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519de23-1d45-4ce3-926d-18699a564948 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:10.434919 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ae1af-c7a5-4bab-aaea-0048e158725f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:10.435578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae1af-c7a5-4bab-aaea-0048e158725f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:10.435897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae1af-c7a5-4bab-aaea-0048e158725f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:10.436197 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae1af-c7a5-4bab-aaea-0048e158725f admin admin] PATH_INFO: `/v3/users/483a1f4cd72047ca86ad0396d5cbc60c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:10.437398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ae1af-c7a5-4bab-aaea-0048e158725f admin admin] RBAC: Authorizing `identity:delete_user(user_id=483a1f4cd72047ca86ad0396d5cbc60c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:10.439524 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ae1af-c7a5-4bab-aaea-0048e158725f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:10.464445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:10.497376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-815ae1af-c7a5-4bab-aaea-0048e158725f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '483a1f4cd72047ca86ad0396d5cbc60c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.498044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-815ae1af-c7a5-4bab-aaea-0048e158725f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '483a1f4cd72047ca86ad0396d5cbc60c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.505080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-815ae1af-c7a5-4bab-aaea-0048e158725f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '483a1f4cd72047ca86ad0396d5cbc60c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.517906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-815ae1af-c7a5-4bab-aaea-0048e158725f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '483a1f4cd72047ca86ad0396d5cbc60c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.518606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-815ae1af-c7a5-4bab-aaea-0048e158725f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '483a1f4cd72047ca86ad0396d5cbc60c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.534163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1204/2398] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:18:10 2022] DELETE /identity/v3/users/483a1f4cd72047ca86ad0396d5cbc60c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:18:10.713047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:10.714977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:10.745363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7b920-5427-4738-9da9-e384bfe8ac07 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:10.804728 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034e09d-0a68-436a-80fd-95ee618067a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:10.805294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034e09d-0a68-436a-80fd-95ee618067a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:10.805569 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034e09d-0a68-436a-80fd-95ee618067a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:10.805839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034e09d-0a68-436a-80fd-95ee618067a5 admin admin] PATH_INFO: `/v3/projects/dd54e6d8cda24bf9ae8874d7be98e92d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:10.807014 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034e09d-0a68-436a-80fd-95ee618067a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd54e6d8cda24bf9ae8874d7be98e92d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:10.808971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034e09d-0a68-436a-80fd-95ee618067a5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:10.845451 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:10.858366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f034e09d-0a68-436a-80fd-95ee618067a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd54e6d8cda24bf9ae8874d7be98e92d', 'request_id': 'req-f034e09d-0a68-436a-80fd-95ee618067a5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.866731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f034e09d-0a68-436a-80fd-95ee618067a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd54e6d8cda24bf9ae8874d7be98e92d', 'request_id': 'req-f034e09d-0a68-436a-80fd-95ee618067a5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.873058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f034e09d-0a68-436a-80fd-95ee618067a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd54e6d8cda24bf9ae8874d7be98e92d', 'request_id': 'req-f034e09d-0a68-436a-80fd-95ee618067a5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.883692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f034e09d-0a68-436a-80fd-95ee618067a5 admin admin] The token cache is being invalidate because project dd54e6d8cda24bf9ae8874d7be98e92d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:18:10.883972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f034e09d-0a68-436a-80fd-95ee618067a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.884599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f034e09d-0a68-436a-80fd-95ee618067a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:10.886141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1195/2399] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:18:10 2022] DELETE /identity/v3/projects/dd54e6d8cda24bf9ae8874d7be98e92d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:18:10.891259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94371ded-4efe-47a0-9854-639a704b2174 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:10.891749 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94371ded-4efe-47a0-9854-639a704b2174 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:10.891866 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94371ded-4efe-47a0-9854-639a704b2174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:10.912471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-94371ded-4efe-47a0-9854-639a704b2174 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:18:10.913714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-94371ded-4efe-47a0-9854-639a704b2174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:10.974914 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1205/2400] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:18:10.984002 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceb0c5-8e65-435d-bd22-09e9b403d415 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:10.984350 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceb0c5-8e65-435d-bd22-09e9b403d415 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:10.984783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceb0c5-8e65-435d-bd22-09e9b403d415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.012787 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6fceb0c5-8e65-435d-bd22-09e9b403d415 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:18:11.014541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6fceb0c5-8e65-435d-bd22-09e9b403d415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:11.025912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1196/2401] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:18:11.030721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94371ded-4efe-47a0-9854-639a704b2174 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:11.041325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94371ded-4efe-47a0-9854-639a704b2174 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:11.046638 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299567b-292e-441f-bb27-21dccf96a09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:11.047245 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299567b-292e-441f-bb27-21dccf96a09b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.047516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299567b-292e-441f-bb27-21dccf96a09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.047786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299567b-292e-441f-bb27-21dccf96a09b admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.048822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299567b-292e-441f-bb27-21dccf96a09b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:18:11.049119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299567b-292e-441f-bb27-21dccf96a09b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:11.050741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299567b-292e-441f-bb27-21dccf96a09b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:11.062600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1206/2402] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:18:11.069423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceb0c5-8e65-435d-bd22-09e9b403d415 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:11.085172 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceb0c5-8e65-435d-bd22-09e9b403d415 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:11.092358 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e096d4-180a-427c-a47f-f42d9a2af99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:11.092989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e096d4-180a-427c-a47f-f42d9a2af99c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.093252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e096d4-180a-427c-a47f-f42d9a2af99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.093517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e096d4-180a-427c-a47f-f42d9a2af99c admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.094151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e096d4-180a-427c-a47f-f42d9a2af99c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:11.096081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e096d4-180a-427c-a47f-f42d9a2af99c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:11.133305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1197/2403] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:18:11 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:18:11.137566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299567b-292e-441f-bb27-21dccf96a09b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:11.147971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299567b-292e-441f-bb27-21dccf96a09b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:11.214445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d943c-8573-4ef7-b59f-a8ba614ca915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:11.215055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d943c-8573-4ef7-b59f-a8ba614ca915 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.215329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d943c-8573-4ef7-b59f-a8ba614ca915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.215593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d943c-8573-4ef7-b59f-a8ba614ca915 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.216223 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d943c-8573-4ef7-b59f-a8ba614ca915 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:11.218175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d943c-8573-4ef7-b59f-a8ba614ca915 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:11.241864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1207/2404] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:18:11 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:18:11.246789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e096d4-180a-427c-a47f-f42d9a2af99c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:11.261683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e096d4-180a-427c-a47f-f42d9a2af99c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:11.267026 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb909a4-d5ee-41b6-a13a-b06df1a05cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:11.267635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb909a4-d5ee-41b6-a13a-b06df1a05cdd admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.267913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb909a4-d5ee-41b6-a13a-b06df1a05cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.268184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb909a4-d5ee-41b6-a13a-b06df1a05cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.268891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb909a4-d5ee-41b6-a13a-b06df1a05cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:11.270616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb909a4-d5ee-41b6-a13a-b06df1a05cdd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:11.281123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1198/2405] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:18:11.287080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d943c-8573-4ef7-b59f-a8ba614ca915 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:11.299289 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d943c-8573-4ef7-b59f-a8ba614ca915 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:11.304267 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e71181-4a2c-463e-9d74-2074626f3949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:11.304901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e71181-4a2c-463e-9d74-2074626f3949 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.305178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e71181-4a2c-463e-9d74-2074626f3949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.305450 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e71181-4a2c-463e-9d74-2074626f3949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.306088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e71181-4a2c-463e-9d74-2074626f3949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:11.307711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e71181-4a2c-463e-9d74-2074626f3949 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:11.318951 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1208/2406] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:18:11.323780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb909a4-d5ee-41b6-a13a-b06df1a05cdd admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:11.339412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb909a4-d5ee-41b6-a13a-b06df1a05cdd admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:11.347295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23551a-a5e9-42b8-b2ed-097a2ef1721a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:11.348231 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23551a-a5e9-42b8-b2ed-097a2ef1721a admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.348642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23551a-a5e9-42b8-b2ed-097a2ef1721a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.349069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23551a-a5e9-42b8-b2ed-097a2ef1721a admin admin] PATH_INFO: `/v3/projects/ec2c6c09002f40ed96bfa2126ab55eeb/users/93e4fc7f34314d2c81ca4f22b9f40243/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.365584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23551a-a5e9-42b8-b2ed-097a2ef1721a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2c6c09002f40ed96bfa2126ab55eeb, user_id=93e4fc7f34314d2c81ca4f22b9f40243, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:11.368249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23551a-a5e9-42b8-b2ed-097a2ef1721a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:11.383612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1199/2407] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:18:11 2022] PUT /identity/v3/projects/ec2c6c09002f40ed96bfa2126ab55eeb/users/93e4fc7f34314d2c81ca4f22b9f40243/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:18:11.395564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d05920-254c-40fb-893c-6b91f3c34811 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.395824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d05920-254c-40fb-893c-6b91f3c34811 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.396092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d05920-254c-40fb-893c-6b91f3c34811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.405031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:11.428562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-22d05920-254c-40fb-893c-6b91f3c34811 None None] MFA Rules not processed for user `93e4fc7f34314d2c81ca4f22b9f40243`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:18:11.429838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-22d05920-254c-40fb-893c-6b91f3c34811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:11.498529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1209/2408] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:18:11.503227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23551a-a5e9-42b8-b2ed-097a2ef1721a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:11.569611 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23551a-a5e9-42b8-b2ed-097a2ef1721a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:11.574947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d8663-4765-45fa-81f5-a6253b2d9604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:11.575547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d8663-4765-45fa-81f5-a6253b2d9604 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.575819 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d8663-4765-45fa-81f5-a6253b2d9604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.576082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d8663-4765-45fa-81f5-a6253b2d9604 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.576743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d8663-4765-45fa-81f5-a6253b2d9604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:11.578649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d8663-4765-45fa-81f5-a6253b2d9604 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:11.618297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1200/2409] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:18:11 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:18:11.623214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d05920-254c-40fb-893c-6b91f3c34811 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:11.634539 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d05920-254c-40fb-893c-6b91f3c34811 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:11.692039 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c0188-622c-45c7-8614-e263f4670025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:11.692638 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c0188-622c-45c7-8614-e263f4670025 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.693059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c0188-622c-45c7-8614-e263f4670025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.693334 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c0188-622c-45c7-8614-e263f4670025 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.693959 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c0188-622c-45c7-8614-e263f4670025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:11.695878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c0188-622c-45c7-8614-e263f4670025 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:11.721561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1210/2410] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:18:11 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:18:11.726126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d8663-4765-45fa-81f5-a6253b2d9604 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:11.737122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d8663-4765-45fa-81f5-a6253b2d9604 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:11.742260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f0114-2548-4700-af3f-170e26d2abb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:11.742854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f0114-2548-4700-af3f-170e26d2abb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.743132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f0114-2548-4700-af3f-170e26d2abb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.743396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f0114-2548-4700-af3f-170e26d2abb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.744028 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f0114-2548-4700-af3f-170e26d2abb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:11.746072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f0114-2548-4700-af3f-170e26d2abb5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:11.757099 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1201/2411] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:18:11.761555 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c0188-622c-45c7-8614-e263f4670025 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:11.772207 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c0188-622c-45c7-8614-e263f4670025 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:11.778473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4a3e9-fabe-4eec-b5cd-bf5273765686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:11.779131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4a3e9-fabe-4eec-b5cd-bf5273765686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.779404 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4a3e9-fabe-4eec-b5cd-bf5273765686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.779671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4a3e9-fabe-4eec-b5cd-bf5273765686 admin admin] PATH_INFO: `/v3/projects/345dfa8aef73493da802069af0584058/users/efba230639884fcf8f2e3b163ff4f52f/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.795782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4a3e9-fabe-4eec-b5cd-bf5273765686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=345dfa8aef73493da802069af0584058, user_id=efba230639884fcf8f2e3b163ff4f52f, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:11.798874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4a3e9-fabe-4eec-b5cd-bf5273765686 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:11.814746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1211/2412] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:18:11 2022] PUT /identity/v3/projects/345dfa8aef73493da802069af0584058/users/efba230639884fcf8f2e3b163ff4f52f/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:18:11.818944 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f0114-2548-4700-af3f-170e26d2abb5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:11.834741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f0114-2548-4700-af3f-170e26d2abb5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:11.867162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:11.900385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8726c-97f8-4ce1-b83a-104ad4423082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:11.901067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8726c-97f8-4ce1-b83a-104ad4423082 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.901349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8726c-97f8-4ce1-b83a-104ad4423082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.901617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8726c-97f8-4ce1-b83a-104ad4423082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.902265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8726c-97f8-4ce1-b83a-104ad4423082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:11.903884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8726c-97f8-4ce1-b83a-104ad4423082 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:11.918487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1202/2413] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:18:11.922986 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4a3e9-fabe-4eec-b5cd-bf5273765686 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:11.934176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4a3e9-fabe-4eec-b5cd-bf5273765686 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:11.939081 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824aff05-6980-4d5a-baac-0dc41c044517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:11.939846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824aff05-6980-4d5a-baac-0dc41c044517 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.940190 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824aff05-6980-4d5a-baac-0dc41c044517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.940526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824aff05-6980-4d5a-baac-0dc41c044517 admin admin] PATH_INFO: `/v3/projects/345dfa8aef73493da802069af0584058/users/efba230639884fcf8f2e3b163ff4f52f/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.942490 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824aff05-6980-4d5a-baac-0dc41c044517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=345dfa8aef73493da802069af0584058, user_id=efba230639884fcf8f2e3b163ff4f52f, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:11.945188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824aff05-6980-4d5a-baac-0dc41c044517 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:11.957225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1212/2414] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:18:11 2022] PUT /identity/v3/projects/345dfa8aef73493da802069af0584058/users/efba230639884fcf8f2e3b163ff4f52f/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:18:11.968018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ed205-2942-45a0-9bb9-27cdce8c18a3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:11.968275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ed205-2942-45a0-9bb9-27cdce8c18a3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:11.968576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ed205-2942-45a0-9bb9-27cdce8c18a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:11.999728 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c68ed205-2942-45a0-9bb9-27cdce8c18a3 None None] MFA Rules not processed for user `efba230639884fcf8f2e3b163ff4f52f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:18:12.000987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c68ed205-2942-45a0-9bb9-27cdce8c18a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:12.086030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1203/2415] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:18:12.097426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824aff05-6980-4d5a-baac-0dc41c044517 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:12.099339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-824aff05-6980-4d5a-baac-0dc41c044517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:12.153601 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:12.163149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824aff05-6980-4d5a-baac-0dc41c044517 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:12.169090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06039410-9591-4205-bcbe-18b76ca565b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:12.169702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06039410-9591-4205-bcbe-18b76ca565b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:12.169984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06039410-9591-4205-bcbe-18b76ca565b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:12.170246 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06039410-9591-4205-bcbe-18b76ca565b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:12.230024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06039410-9591-4205-bcbe-18b76ca565b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:12.231277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06039410-9591-4205-bcbe-18b76ca565b4 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:12.241396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1213/2416] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:18:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:18:15.906537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ed205-2942-45a0-9bb9-27cdce8c18a3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:15.979908 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ed205-2942-45a0-9bb9-27cdce8c18a3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:15.984628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:15.985284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:15.985559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:15.985824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] PATH_INFO: `/v3/users/93e4fc7f34314d2c81ca4f22b9f40243` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:15.986948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93e4fc7f34314d2c81ca4f22b9f40243)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:15.988921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:16.045694 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93e4fc7f34314d2c81ca4f22b9f40243'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:16.046367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93e4fc7f34314d2c81ca4f22b9f40243'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:16.052582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:16.055143 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93e4fc7f34314d2c81ca4f22b9f40243'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:16.088237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93e4fc7f34314d2c81ca4f22b9f40243'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:16.089378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93e4fc7f34314d2c81ca4f22b9f40243'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:16.116683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1204/2417] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:18:15 2022] DELETE /identity/v3/users/93e4fc7f34314d2c81ca4f22b9f40243 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:18:16.124547 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b472c0-0429-4a63-8424-d03d3c9ce195 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:16.127297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b472c0-0429-4a63-8424-d03d3c9ce195 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:16.127297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b472c0-0429-4a63-8424-d03d3c9ce195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:16.149824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f7b472c0-0429-4a63-8424-d03d3c9ce195 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:18:16.151723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f7b472c0-0429-4a63-8424-d03d3c9ce195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:16.239190 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1214/2418] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:18:16.249600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:16.251106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:16.330537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c8f98-a383-444c-a5d6-3d36c3f2b934 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:16.343560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f364-8653-4a42-93aa-b13c716d6cea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:16.343560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f364-8653-4a42-93aa-b13c716d6cea service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:16.343560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f364-8653-4a42-93aa-b13c716d6cea service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:16.343560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f364-8653-4a42-93aa-b13c716d6cea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:16.357365 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66f364-8653-4a42-93aa-b13c716d6cea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:16.359016 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66f364-8653-4a42-93aa-b13c716d6cea service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:16.372068 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1205/2419] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:18:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:18:16.718038 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b472c0-0429-4a63-8424-d03d3c9ce195 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:16.719676 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f7b472c0-0429-4a63-8424-d03d3c9ce195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:16.734145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b472c0-0429-4a63-8424-d03d3c9ce195 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:16.740000 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:16.740761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:16.740967 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:16.741239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] PATH_INFO: `/v3/projects/ec2c6c09002f40ed96bfa2126ab55eeb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:16.743641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2c6c09002f40ed96bfa2126ab55eeb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:16.746838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:16.823161 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2c6c09002f40ed96bfa2126ab55eeb', 'request_id': 'req-d25b8f48-14d6-475a-8d01-a7dd860b1b91'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:16.835779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2c6c09002f40ed96bfa2126ab55eeb', 'request_id': 'req-d25b8f48-14d6-475a-8d01-a7dd860b1b91'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:16.843056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2c6c09002f40ed96bfa2126ab55eeb', 'request_id': 'req-d25b8f48-14d6-475a-8d01-a7dd860b1b91'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:16.857982 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] The token cache is being invalidate because project ec2c6c09002f40ed96bfa2126ab55eeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:18:16.858280 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:16.858881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:16.860637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1215/2420] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:18:16 2022] DELETE /identity/v3/projects/ec2c6c09002f40ed96bfa2126ab55eeb => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:18:16.864937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f364-8653-4a42-93aa-b13c716d6cea service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:16.875424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f364-8653-4a42-93aa-b13c716d6cea service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:16.944979 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:16.945859 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:16.946274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:16.946274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] PATH_INFO: `/v3/users/efba230639884fcf8f2e3b163ff4f52f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:16.947547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] RBAC: Authorizing `identity:delete_user(user_id=efba230639884fcf8f2e3b163ff4f52f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:16.949766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:17.002065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efba230639884fcf8f2e3b163ff4f52f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:17.002827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efba230639884fcf8f2e3b163ff4f52f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:17.010181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efba230639884fcf8f2e3b163ff4f52f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:17.022632 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efba230639884fcf8f2e3b163ff4f52f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:17.023339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efba230639884fcf8f2e3b163ff4f52f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:17.026207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:17.041168 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1206/2421] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:18:16 2022] DELETE /identity/v3/users/efba230639884fcf8f2e3b163ff4f52f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:18:17.348682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:17.350948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:17.380201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b8f48-14d6-475a-8d01-a7dd860b1b91 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:17.421862 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:17.469836 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792baf5-7d54-45a1-9161-df9b9f2aedf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:17.470487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792baf5-7d54-45a1-9161-df9b9f2aedf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:17.470767 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792baf5-7d54-45a1-9161-df9b9f2aedf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:17.471033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792baf5-7d54-45a1-9161-df9b9f2aedf3 admin admin] PATH_INFO: `/v3/projects/345dfa8aef73493da802069af0584058` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:17.472192 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792baf5-7d54-45a1-9161-df9b9f2aedf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=345dfa8aef73493da802069af0584058)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:17.474192 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792baf5-7d54-45a1-9161-df9b9f2aedf3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:17.524913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f792baf5-7d54-45a1-9161-df9b9f2aedf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345dfa8aef73493da802069af0584058', 'request_id': 'req-f792baf5-7d54-45a1-9161-df9b9f2aedf3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:17.531721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f792baf5-7d54-45a1-9161-df9b9f2aedf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345dfa8aef73493da802069af0584058', 'request_id': 'req-f792baf5-7d54-45a1-9161-df9b9f2aedf3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:17.537818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f792baf5-7d54-45a1-9161-df9b9f2aedf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345dfa8aef73493da802069af0584058', 'request_id': 'req-f792baf5-7d54-45a1-9161-df9b9f2aedf3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:17.551584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f792baf5-7d54-45a1-9161-df9b9f2aedf3 admin admin] The token cache is being invalidate because project 345dfa8aef73493da802069af0584058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:18:17.551876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f792baf5-7d54-45a1-9161-df9b9f2aedf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:17.552514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f792baf5-7d54-45a1-9161-df9b9f2aedf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:18:17.554217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1216/2422] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:18:17 2022] DELETE /identity/v3/projects/345dfa8aef73493da802069af0584058 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:18:27.039309 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:27.548579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:27.824575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:27.826330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:27.886984 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b2142-acc0-403f-8e9c-fbbb304f805d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:27.940468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbf5fe-6569-4163-96ef-bc5564233233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:27.941110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbf5fe-6569-4163-96ef-bc5564233233 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:27.941383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbf5fe-6569-4163-96ef-bc5564233233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:27.941649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbf5fe-6569-4163-96ef-bc5564233233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:27.943668 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e1dbf5fe-6569-4163-96ef-bc5564233233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:28.009086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbf5fe-6569-4163-96ef-bc5564233233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:28.010740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbf5fe-6569-4163-96ef-bc5564233233 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:28.020961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1207/2423] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:18:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:18:29.156755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99b196-2af0-43e4-b7da-297d38fe941f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:29.157339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99b196-2af0-43e4-b7da-297d38fe941f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:29.157339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99b196-2af0-43e4-b7da-297d38fe941f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:29.178231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fd99b196-2af0-43e4-b7da-297d38fe941f None None] MFA Rules not processed for user `71e0c39501bf48c2a63257a4cf75def1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:18:29.179515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fd99b196-2af0-43e4-b7da-297d38fe941f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:29.256801 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1217/2424] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:18:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:18:38.016192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=70 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:39.247056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=25 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:41.918790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbf5fe-6569-4163-96ef-bc5564233233 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:41.920258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e1dbf5fe-6569-4163-96ef-bc5564233233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:41.932130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbf5fe-6569-4163-96ef-bc5564233233 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:41.936833 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcd08a-22f2-4b06-9cfc-1721cc427fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:41.937547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd08a-22f2-4b06-9cfc-1721cc427fea service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:41.937822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd08a-22f2-4b06-9cfc-1721cc427fea service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:41.938093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd08a-22f2-4b06-9cfc-1721cc427fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:41.939995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8adcd08a-22f2-4b06-9cfc-1721cc427fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:42.023622 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcd08a-22f2-4b06-9cfc-1721cc427fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:42.025257 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcd08a-22f2-4b06-9cfc-1721cc427fea service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:42.037570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1208/2425] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:18:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:18:43.262651 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99b196-2af0-43e4-b7da-297d38fe941f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:43.266794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fd99b196-2af0-43e4-b7da-297d38fe941f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:43.286204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99b196-2af0-43e4-b7da-297d38fe941f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:43.290499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de9c7c-097a-485a-99df-404fbcb82fa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:43.291339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de9c7c-097a-485a-99df-404fbcb82fa3 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:43.291701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de9c7c-097a-485a-99df-404fbcb82fa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:43.292146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de9c7c-097a-485a-99df-404fbcb82fa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:43.294812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-19de9c7c-097a-485a-99df-404fbcb82fa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:43.313302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de9c7c-097a-485a-99df-404fbcb82fa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:43.314724 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de9c7c-097a-485a-99df-404fbcb82fa3 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:43.329760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1218/2426] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:18:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:18:52.031851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=34 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:52.704777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcd08a-22f2-4b06-9cfc-1721cc427fea service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:52.706411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8adcd08a-22f2-4b06-9cfc-1721cc427fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:52.769594 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcd08a-22f2-4b06-9cfc-1721cc427fea service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:52.773742 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833bc3d-731e-4d93-86f7-d3e8b00ca992 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a76937ab52e445c59b9eb9638fd4a753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:52.774351 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833bc3d-731e-4d93-86f7-d3e8b00ca992 service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:52.774627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833bc3d-731e-4d93-86f7-d3e8b00ca992 service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:52.774898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833bc3d-731e-4d93-86f7-d3e8b00ca992 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:52.777765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c833bc3d-731e-4d93-86f7-d3e8b00ca992 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:52.841471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833bc3d-731e-4d93-86f7-d3e8b00ca992 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:52.842620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833bc3d-731e-4d93-86f7-d3e8b00ca992 service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:52.851968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1209/2427] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:18:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:18:52.993254 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6daef-d633-4819-b8b9-5011e4a49439 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:52.993764 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6daef-d633-4819-b8b9-5011e4a49439 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:52.993944 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6daef-d633-4819-b8b9-5011e4a49439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:53.015308 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-08d6daef-d633-4819-b8b9-5011e4a49439 None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:18:53.016597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-08d6daef-d633-4819-b8b9-5011e4a49439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:53.077424 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1219/2428] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:18:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:18:53.126952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833bc3d-731e-4d93-86f7-d3e8b00ca992 service glance] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:53.128382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c833bc3d-731e-4d93-86f7-d3e8b00ca992 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:53.215752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833bc3d-731e-4d93-86f7-d3e8b00ca992 service glance] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:53.221223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b2aa1-9349-4a7d-aa7c-8945a0cc67ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:53.221843 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b2aa1-9349-4a7d-aa7c-8945a0cc67ca service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:53.222122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b2aa1-9349-4a7d-aa7c-8945a0cc67ca service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:53.222387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b2aa1-9349-4a7d-aa7c-8945a0cc67ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:53.231228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b2aa1-9349-4a7d-aa7c-8945a0cc67ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:53.232882 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b2aa1-9349-4a7d-aa7c-8945a0cc67ca service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:53.241611 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1210/2429] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:18:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:18:53.725904 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6daef-d633-4819-b8b9-5011e4a49439 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:53.741278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6daef-d633-4819-b8b9-5011e4a49439 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:53.761852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38780c41-b0f2-432e-a486-9c9ed37ee076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:53.762590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38780c41-b0f2-432e-a486-9c9ed37ee076 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:53.762875 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38780c41-b0f2-432e-a486-9c9ed37ee076 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:53.763176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38780c41-b0f2-432e-a486-9c9ed37ee076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:53.765580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-38780c41-b0f2-432e-a486-9c9ed37ee076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:53.852960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38780c41-b0f2-432e-a486-9c9ed37ee076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:53.856677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38780c41-b0f2-432e-a486-9c9ed37ee076 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:53.872064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1220/2430] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:18:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:18:54.929553 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b2aa1-9349-4a7d-aa7c-8945a0cc67ca service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:18:54.931805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-526b2aa1-9349-4a7d-aa7c-8945a0cc67ca service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:54.953623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b2aa1-9349-4a7d-aa7c-8945a0cc67ca service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:18:54.962282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db053caa-0f61-4ed6-b9b1-b4b9412a8196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:18:54.962902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db053caa-0f61-4ed6-b9b1-b4b9412a8196 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:18:54.963192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db053caa-0f61-4ed6-b9b1-b4b9412a8196 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:18:54.963906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db053caa-0f61-4ed6-b9b1-b4b9412a8196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:18:54.966779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-db053caa-0f61-4ed6-b9b1-b4b9412a8196 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:18:55.037389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:18:55.084223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db053caa-0f61-4ed6-b9b1-b4b9412a8196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:18:55.086653 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db053caa-0f61-4ed6-b9b1-b4b9412a8196 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:18:55.107483 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1211/2431] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:18:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:19:03.865360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=61 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:19:05.095323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:19:10.600950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38780c41-b0f2-432e-a486-9c9ed37ee076 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:19:10.610199 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38780c41-b0f2-432e-a486-9c9ed37ee076 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:19:10.614640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a0628-443b-49f0-87ad-4c15b736a7c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:19:10.615242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a0628-443b-49f0-87ad-4c15b736a7c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:10.615523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a0628-443b-49f0-87ad-4c15b736a7c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:10.615794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a0628-443b-49f0-87ad-4c15b736a7c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:10.618141 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-872a0628-443b-49f0-87ad-4c15b736a7c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:19:10.629067 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a0628-443b-49f0-87ad-4c15b736a7c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:19:10.630488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a0628-443b-49f0-87ad-4c15b736a7c0 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:19:10.639480 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1221/2432] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:19:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:19:14.465879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db053caa-0f61-4ed6-b9b1-b4b9412a8196 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:19:14.476390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db053caa-0f61-4ed6-b9b1-b4b9412a8196 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:19:14.480642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b28648-18f1-461b-bca8-05b3b2087248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:19:14.481145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b28648-18f1-461b-bca8-05b3b2087248 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:14.481407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b28648-18f1-461b-bca8-05b3b2087248 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:14.482609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b28648-18f1-461b-bca8-05b3b2087248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:14.483481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c1b28648-18f1-461b-bca8-05b3b2087248 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:19:14.498229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b28648-18f1-461b-bca8-05b3b2087248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:19:14.500498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b28648-18f1-461b-bca8-05b3b2087248 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:19:14.509932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1212/2433] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:19:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:19:14.656625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13859326-e217-45bf-893c-c2222195034e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:14.657261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13859326-e217-45bf-893c-c2222195034e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:14.657261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13859326-e217-45bf-893c-c2222195034e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:14.658537 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1222/2434] 10.222.0.79 () {54 vars in 921 bytes} [Tue Sep 6 21:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:19:14.664389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a888af7-3f08-41c8-8acd-4f15ac97fca2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:14.664807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a888af7-3f08-41c8-8acd-4f15ac97fca2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:14.665152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a888af7-3f08-41c8-8acd-4f15ac97fca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:14.686333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5a888af7-3f08-41c8-8acd-4f15ac97fca2 None None] MFA Rules not processed for user `12d51454d20c46d7a5e6d963a2f987cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:19:14.687649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5a888af7-3f08-41c8-8acd-4f15ac97fca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:19:14.697550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1213/2435] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Sep 6 21:19:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:19:14.709617 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13859326-e217-45bf-893c-c2222195034e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:19:14.721421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13859326-e217-45bf-893c-c2222195034e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:19:14.727909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a8154-7618-4a81-93f6-65c5fa01c185 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a76937ab52e445c59b9eb9638fd4a753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:19:14.728602 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a8154-7618-4a81-93f6-65c5fa01c185 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:14.728925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a8154-7618-4a81-93f6-65c5fa01c185 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:14.729191 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a8154-7618-4a81-93f6-65c5fa01c185 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:14.740906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a8154-7618-4a81-93f6-65c5fa01c185 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:19:14.742392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a8154-7618-4a81-93f6-65c5fa01c185 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:19:14.753267 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1223/2436] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:19:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:19:14.827020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58bd13-4a7b-4aaa-a92e-d1e0efefcdc8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:14.827317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58bd13-4a7b-4aaa-a92e-d1e0efefcdc8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:14.827633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58bd13-4a7b-4aaa-a92e-d1e0efefcdc8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:14.829066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1214/2437] 10.222.0.79 () {54 vars in 921 bytes} [Tue Sep 6 21:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:19:14.834047 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d013b3c-71d0-4f10-80fe-0ce7c6208482 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:14.834333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d013b3c-71d0-4f10-80fe-0ce7c6208482 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:14.834765 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d013b3c-71d0-4f10-80fe-0ce7c6208482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:14.869441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0d013b3c-71d0-4f10-80fe-0ce7c6208482 None None] MFA Rules not processed for user `d5a45dec40844877a6b5b301fc44a110`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:19:14.870853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0d013b3c-71d0-4f10-80fe-0ce7c6208482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:19:14.878214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1224/2438] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Sep 6 21:19:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:19:14.891616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58bd13-4a7b-4aaa-a92e-d1e0efefcdc8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:19:14.903466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58bd13-4a7b-4aaa-a92e-d1e0efefcdc8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:19:14.907157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2f652-e13d-4176-a559-f5a72e4bd45b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:19:14.907932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2f652-e13d-4176-a559-f5a72e4bd45b service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:14.908202 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2f652-e13d-4176-a559-f5a72e4bd45b service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:14.908469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2f652-e13d-4176-a559-f5a72e4bd45b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:14.916475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2f652-e13d-4176-a559-f5a72e4bd45b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:19:14.917977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2f652-e13d-4176-a559-f5a72e4bd45b service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:19:14.928070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1215/2439] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:19:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:19:15.046517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda23f9-4ce6-4107-b051-98193aaf385d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:15.046807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda23f9-4ce6-4107-b051-98193aaf385d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:15.047102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda23f9-4ce6-4107-b051-98193aaf385d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:15.048841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1225/2440] 10.222.0.79 () {54 vars in 921 bytes} [Tue Sep 6 21:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:19:15.054348 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752422b-0c39-446a-9e5f-fddf92495237 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:15.054684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752422b-0c39-446a-9e5f-fddf92495237 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:15.054949 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752422b-0c39-446a-9e5f-fddf92495237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:15.075340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1752422b-0c39-446a-9e5f-fddf92495237 None None] MFA Rules not processed for user `12d51454d20c46d7a5e6d963a2f987cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:19:15.076541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1752422b-0c39-446a-9e5f-fddf92495237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:19:15.086363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1216/2441] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Sep 6 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:19:15.097161 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda23f9-4ce6-4107-b051-98193aaf385d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:19:15.108304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda23f9-4ce6-4107-b051-98193aaf385d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:19:15.111851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c3fe7-f43f-43eb-9a06-23788f3e38ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:19:15.112445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c3fe7-f43f-43eb-9a06-23788f3e38ee service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:15.112776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c3fe7-f43f-43eb-9a06-23788f3e38ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:15.113053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c3fe7-f43f-43eb-9a06-23788f3e38ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:15.123398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c3fe7-f43f-43eb-9a06-23788f3e38ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:19:15.124877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c3fe7-f43f-43eb-9a06-23788f3e38ee service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:19:15.136040 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1226/2442] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:19:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:19:15.206616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738edd07-fc73-49bb-a36e-034630fb25dc None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:15.206925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738edd07-fc73-49bb-a36e-034630fb25dc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:15.207196 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738edd07-fc73-49bb-a36e-034630fb25dc None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:15.208547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1217/2443] 10.222.0.79 () {54 vars in 961 bytes} [Tue Sep 6 21:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:19:15.213890 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f680a0-3686-4731-9672-4a3b26cbefd7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:15.214178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f680a0-3686-4731-9672-4a3b26cbefd7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:15.214448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f680a0-3686-4731-9672-4a3b26cbefd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:15.236423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d2f680a0-3686-4731-9672-4a3b26cbefd7 None None] MFA Rules not processed for user `c40b5ff21d1940749bf479cc936fc0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:19:15.237702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d2f680a0-3686-4731-9672-4a3b26cbefd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:19:15.309416 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1227/2444] 10.222.0.79 () {58 vars in 1058 bytes} [Tue Sep 6 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:19:15.317192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738edd07-fc73-49bb-a36e-034630fb25dc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:19:15.331490 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738edd07-fc73-49bb-a36e-034630fb25dc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:19:15.337802 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e6932-c646-4052-81a8-02926d8fcb36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:19:15.338380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e6932-c646-4052-81a8-02926d8fcb36 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:15.338654 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e6932-c646-4052-81a8-02926d8fcb36 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:15.338920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e6932-c646-4052-81a8-02926d8fcb36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:15.340829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5d5e6932-c646-4052-81a8-02926d8fcb36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:19:15.353279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e6932-c646-4052-81a8-02926d8fcb36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:19:15.354844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e6932-c646-4052-81a8-02926d8fcb36 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:19:15.365294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1218/2445] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:19:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:19:21.748729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f680a0-3686-4731-9672-4a3b26cbefd7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:19:21.763787 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f680a0-3686-4731-9672-4a3b26cbefd7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:19:21.769326 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d3239-587a-436b-aa96-cc0562b928b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:19:21.769901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d3239-587a-436b-aa96-cc0562b928b9 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:21.770181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d3239-587a-436b-aa96-cc0562b928b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:21.770454 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d3239-587a-436b-aa96-cc0562b928b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:21.772266 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a80d3239-587a-436b-aa96-cc0562b928b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:19:21.786396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d3239-587a-436b-aa96-cc0562b928b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:19:21.788353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d3239-587a-436b-aa96-cc0562b928b9 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:19:21.800313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1228/2446] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:19:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:19:22.236704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c3608-a0cb-4e85-9e60-7b85bfe7da62 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:22.237186 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c3608-a0cb-4e85-9e60-7b85bfe7da62 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:22.237375 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c3608-a0cb-4e85-9e60-7b85bfe7da62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:22.257385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5f3c3608-a0cb-4e85-9e60-7b85bfe7da62 None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:19:22.258648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5f3c3608-a0cb-4e85-9e60-7b85bfe7da62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:19:22.267091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1219/2447] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:19:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:19:22.280148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d3239-587a-436b-aa96-cc0562b928b9 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:19:22.292287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d3239-587a-436b-aa96-cc0562b928b9 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:19:22.298461 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812e3ee-5460-469b-923b-87a805415e5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:19:22.299073 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812e3ee-5460-469b-923b-87a805415e5c service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:19:22.299353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812e3ee-5460-469b-923b-87a805415e5c service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:19:22.299624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812e3ee-5460-469b-923b-87a805415e5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:19:22.308521 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812e3ee-5460-469b-923b-87a805415e5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:19:22.310269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812e3ee-5460-469b-923b-87a805415e5c service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:19:22.319254 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1229/2448] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:19:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:19:32.252212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=27 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:19:32.315645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=62 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:20:02.543297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c3608-a0cb-4e85-9e60-7b85bfe7da62 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:20:02.557884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c3608-a0cb-4e85-9e60-7b85bfe7da62 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:20:02.564426 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04966b47-fc9e-4682-ae50-28a8a864a482 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:20:02.565058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04966b47-fc9e-4682-ae50-28a8a864a482 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:20:02.565320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04966b47-fc9e-4682-ae50-28a8a864a482 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:20:02.565593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04966b47-fc9e-4682-ae50-28a8a864a482 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:20:02.567530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-04966b47-fc9e-4682-ae50-28a8a864a482 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:20:02.631854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04966b47-fc9e-4682-ae50-28a8a864a482 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:20:02.633583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04966b47-fc9e-4682-ae50-28a8a864a482 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:20:02.643574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1220/2449] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:20:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:20:03.201209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7e34c-27c4-4246-9703-77b55d715a4a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:20:03.202181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7e34c-27c4-4246-9703-77b55d715a4a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:20:03.202451 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7e34c-27c4-4246-9703-77b55d715a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:20:03.226576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-45f7e34c-27c4-4246-9703-77b55d715a4a None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:20:03.228484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-45f7e34c-27c4-4246-9703-77b55d715a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:20:03.237148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1230/2450] 10.222.0.79 () {58 vars in 967 bytes} [Tue Sep 6 21:20:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:20:03.267121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04966b47-fc9e-4682-ae50-28a8a864a482 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:20:03.286042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04966b47-fc9e-4682-ae50-28a8a864a482 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:20:03.293773 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387171f9-fd81-4d67-b141-11c67ccd0c6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:20:03.294588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387171f9-fd81-4d67-b141-11c67ccd0c6e service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:20:03.294948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387171f9-fd81-4d67-b141-11c67ccd0c6e service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:20:03.295317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387171f9-fd81-4d67-b141-11c67ccd0c6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:20:03.306306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387171f9-fd81-4d67-b141-11c67ccd0c6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:20:03.308265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387171f9-fd81-4d67-b141-11c67ccd0c6e service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:20:03.318038 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1221/2451] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:20:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:20:10.282751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7e34c-27c4-4246-9703-77b55d715a4a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:20:10.294736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7e34c-27c4-4246-9703-77b55d715a4a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:20:10.300531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd1405-7fe9-4de7-9343-0c5d35e5bc3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:20:10.301163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd1405-7fe9-4de7-9343-0c5d35e5bc3b service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:20:10.301434 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd1405-7fe9-4de7-9343-0c5d35e5bc3b service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:20:10.301699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd1405-7fe9-4de7-9343-0c5d35e5bc3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:20:10.303497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-71cd1405-7fe9-4de7-9343-0c5d35e5bc3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:20:10.315975 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd1405-7fe9-4de7-9343-0c5d35e5bc3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:20:10.317629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd1405-7fe9-4de7-9343-0c5d35e5bc3b service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:20:10.332312 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1231/2452] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:20:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:20:10.356049 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387171f9-fd81-4d67-b141-11c67ccd0c6e service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:20:10.358411 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-387171f9-fd81-4d67-b141-11c67ccd0c6e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:20:10.444080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387171f9-fd81-4d67-b141-11c67ccd0c6e service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:20:10.450030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4bf4e-cc04-422b-921d-17776a86dec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:20:10.450671 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4bf4e-cc04-422b-921d-17776a86dec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:20:10.450946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4bf4e-cc04-422b-921d-17776a86dec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:20:10.451210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4bf4e-cc04-422b-921d-17776a86dec1 admin admin] PATH_INFO: `/v3/users/71e0c39501bf48c2a63257a4cf75def1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:20:10.452422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4bf4e-cc04-422b-921d-17776a86dec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71e0c39501bf48c2a63257a4cf75def1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:20:10.454421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4bf4e-cc04-422b-921d-17776a86dec1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:20:10.505049 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dfe4bf4e-cc04-422b-921d-17776a86dec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71e0c39501bf48c2a63257a4cf75def1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:20:10.505705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dfe4bf4e-cc04-422b-921d-17776a86dec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71e0c39501bf48c2a63257a4cf75def1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:20:10.512596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dfe4bf4e-cc04-422b-921d-17776a86dec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71e0c39501bf48c2a63257a4cf75def1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:20:10.524392 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dfe4bf4e-cc04-422b-921d-17776a86dec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71e0c39501bf48c2a63257a4cf75def1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:20:10.525065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dfe4bf4e-cc04-422b-921d-17776a86dec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71e0c39501bf48c2a63257a4cf75def1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:20:10.538858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1222/2453] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:20:10 2022] DELETE /identity/v3/users/71e0c39501bf48c2a63257a4cf75def1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:20:10.717384 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd1405-7fe9-4de7-9343-0c5d35e5bc3b service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:20:10.728306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd1405-7fe9-4de7-9343-0c5d35e5bc3b service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:20:10.783566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:20:10.784201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:20:10.784476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:20:10.784768 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] PATH_INFO: `/v3/projects/3c0901a228de4c59bc0e999f55bc1e08` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:20:10.785950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c0901a228de4c59bc0e999f55bc1e08)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:20:10.787868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:20:10.842888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c0901a228de4c59bc0e999f55bc1e08', 'request_id': 'req-8adace55-d138-4cba-9c3b-f52e937f9b0c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:20:10.850213 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c0901a228de4c59bc0e999f55bc1e08', 'request_id': 'req-8adace55-d138-4cba-9c3b-f52e937f9b0c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:20:10.858726 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c0901a228de4c59bc0e999f55bc1e08', 'request_id': 'req-8adace55-d138-4cba-9c3b-f52e937f9b0c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:20:10.878804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] The token cache is being invalidate because project 3c0901a228de4c59bc0e999f55bc1e08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:20:10.879171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:20:10.879773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:20:10.881549 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1232/2454] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:20:10 2022] DELETE /identity/v3/projects/3c0901a228de4c59bc0e999f55bc1e08 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:20:20.535541 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:20:20.871908 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:20:28.078627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786d08e-d12a-4aa3-b394-f2e9c993e595 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:20:28.079183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786d08e-d12a-4aa3-b394-f2e9c993e595 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:20:28.079183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786d08e-d12a-4aa3-b394-f2e9c993e595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:20:28.100606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a786d08e-d12a-4aa3-b394-f2e9c993e595 None None] MFA Rules not processed for user `2ec47d6e2e9b40a6bd207c084041f7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:20:28.101848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a786d08e-d12a-4aa3-b394-f2e9c993e595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:20:28.164526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1223/2455] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:20:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:20:38.156573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=25 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:21:16.119512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:21:16.121466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:21:16.218248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adace55-d138-4cba-9c3b-f52e937f9b0c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:21:16.225949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d40ee-80c3-4a4a-b0af-1ab4697dc6b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:21:16.226671 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d40ee-80c3-4a4a-b0af-1ab4697dc6b6 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:21:16.226955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d40ee-80c3-4a4a-b0af-1ab4697dc6b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:21:16.227321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d40ee-80c3-4a4a-b0af-1ab4697dc6b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:21:16.229200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b09d40ee-80c3-4a4a-b0af-1ab4697dc6b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:21:16.242248 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d40ee-80c3-4a4a-b0af-1ab4697dc6b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:21:16.244163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d40ee-80c3-4a4a-b0af-1ab4697dc6b6 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:21:16.256714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1233/2456] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:21:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:21:20.914435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d8b94-c7f1-474e-9be2-883fd94cb16d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:21:20.915008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d8b94-c7f1-474e-9be2-883fd94cb16d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:21:20.915008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d8b94-c7f1-474e-9be2-883fd94cb16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:21:20.936419 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-199d8b94-c7f1-474e-9be2-883fd94cb16d None None] MFA Rules not processed for user `54fe3d99f05c45f099804fccb4679503`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:21:20.937664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-199d8b94-c7f1-474e-9be2-883fd94cb16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:21:21.133844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1224/2457] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:21:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:21:21.464531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d40ee-80c3-4a4a-b0af-1ab4697dc6b6 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:21:21.466293 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b09d40ee-80c3-4a4a-b0af-1ab4697dc6b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:21:21.528073 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d40ee-80c3-4a4a-b0af-1ab4697dc6b6 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:21:21.531733 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebf455-75ce-49dc-9232-c729f0c99c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:21:21.532341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebf455-75ce-49dc-9232-c729f0c99c45 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:21:21.532614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebf455-75ce-49dc-9232-c729f0c99c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:21:21.532918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebf455-75ce-49dc-9232-c729f0c99c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:21:21.534627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-58ebf455-75ce-49dc-9232-c729f0c99c45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:21:21.544778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebf455-75ce-49dc-9232-c729f0c99c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:21:21.546217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebf455-75ce-49dc-9232-c729f0c99c45 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:21:21.554973 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1234/2458] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:21:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:21:29.822393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d8b94-c7f1-474e-9be2-883fd94cb16d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:21:29.836834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d8b94-c7f1-474e-9be2-883fd94cb16d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:21:29.843174 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22244a98-8a07-4afb-992e-427b284e199b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:21:29.843726 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22244a98-8a07-4afb-992e-427b284e199b service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:21:29.843999 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22244a98-8a07-4afb-992e-427b284e199b service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:21:29.844396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22244a98-8a07-4afb-992e-427b284e199b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:21:29.846237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-22244a98-8a07-4afb-992e-427b284e199b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:21:29.860235 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22244a98-8a07-4afb-992e-427b284e199b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:21:29.861870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22244a98-8a07-4afb-992e-427b284e199b service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:21:29.872990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1225/2459] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:21:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:21:31.551099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=60 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:21:39.868471 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=31 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:21:47.568880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18286ce-9ce8-45e1-a5a2-b621690928bb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:21:47.569474 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18286ce-9ce8-45e1-a5a2-b621690928bb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:21:47.569474 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18286ce-9ce8-45e1-a5a2-b621690928bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:21:47.609496 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d18286ce-9ce8-45e1-a5a2-b621690928bb None None] MFA Rules not processed for user `6d0c1c510b0a4fa39927f7f92352d034`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:21:47.613366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d18286ce-9ce8-45e1-a5a2-b621690928bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:21:47.708009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1235/2460] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:21:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:21:57.705643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=37 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:21:58.374094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22244a98-8a07-4afb-992e-427b284e199b service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:21:58.394698 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22244a98-8a07-4afb-992e-427b284e199b service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:21:58.398451 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19fd7a-bd17-41a2-8a6d-572d68efd1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:21:58.399045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19fd7a-bd17-41a2-8a6d-572d68efd1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:21:58.399314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19fd7a-bd17-41a2-8a6d-572d68efd1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:21:58.400381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19fd7a-bd17-41a2-8a6d-572d68efd1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:21:58.401689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3f19fd7a-bd17-41a2-8a6d-572d68efd1c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:21:58.509701 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19fd7a-bd17-41a2-8a6d-572d68efd1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:21:58.511083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19fd7a-bd17-41a2-8a6d-572d68efd1c0 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:21:58.531279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1226/2461] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:22:02.290918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18286ce-9ce8-45e1-a5a2-b621690928bb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:02.292330 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d18286ce-9ce8-45e1-a5a2-b621690928bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:02.422780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18286ce-9ce8-45e1-a5a2-b621690928bb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:02.428277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe9959-b90c-42fc-b4d0-090661ec8cc7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:02.439488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe9959-b90c-42fc-b4d0-090661ec8cc7 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:02.439779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe9959-b90c-42fc-b4d0-090661ec8cc7 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:02.440046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe9959-b90c-42fc-b4d0-090661ec8cc7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:02.441096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19fd7a-bd17-41a2-8a6d-572d68efd1c0 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:02.442396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fcbe9959-b90c-42fc-b4d0-090661ec8cc7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:02.444084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3f19fd7a-bd17-41a2-8a6d-572d68efd1c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:02.461089 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19fd7a-bd17-41a2-8a6d-572d68efd1c0 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:02.463752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe9959-b90c-42fc-b4d0-090661ec8cc7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:02.465720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe9959-b90c-42fc-b4d0-090661ec8cc7 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:02.467154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec88d7-8ba2-4f68-9cbb-857b756f0559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:02.468007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec88d7-8ba2-4f68-9cbb-857b756f0559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:02.468450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec88d7-8ba2-4f68-9cbb-857b756f0559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:02.468913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec88d7-8ba2-4f68-9cbb-857b756f0559 admin admin] PATH_INFO: `/v3/users/2ec47d6e2e9b40a6bd207c084041f7a9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:02.470576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec88d7-8ba2-4f68-9cbb-857b756f0559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ec47d6e2e9b40a6bd207c084041f7a9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:02.473201 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec88d7-8ba2-4f68-9cbb-857b756f0559 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:02.483375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1236/2462] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:22:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:22:02.530019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-eaec88d7-8ba2-4f68-9cbb-857b756f0559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ec47d6e2e9b40a6bd207c084041f7a9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:22:02.531113 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-eaec88d7-8ba2-4f68-9cbb-857b756f0559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ec47d6e2e9b40a6bd207c084041f7a9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:22:02.539239 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-eaec88d7-8ba2-4f68-9cbb-857b756f0559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ec47d6e2e9b40a6bd207c084041f7a9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:22:02.555583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-eaec88d7-8ba2-4f68-9cbb-857b756f0559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ec47d6e2e9b40a6bd207c084041f7a9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:22:02.556599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-eaec88d7-8ba2-4f68-9cbb-857b756f0559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ec47d6e2e9b40a6bd207c084041f7a9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:22:02.576723 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1227/2463] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:22:02 2022] DELETE /identity/v3/users/2ec47d6e2e9b40a6bd207c084041f7a9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:22:02.782482 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe9959-b90c-42fc-b4d0-090661ec8cc7 service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:02.796981 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe9959-b90c-42fc-b4d0-090661ec8cc7 service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:02.856606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a222f1-e1b4-4147-8ca9-86488f11bf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:02.857175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a222f1-e1b4-4147-8ca9-86488f11bf55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:02.857448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a222f1-e1b4-4147-8ca9-86488f11bf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:02.857713 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a222f1-e1b4-4147-8ca9-86488f11bf55 admin admin] PATH_INFO: `/v3/projects/ff397d6b3f4a42bf8b68b3bc55600dba` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:02.858953 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a222f1-e1b4-4147-8ca9-86488f11bf55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff397d6b3f4a42bf8b68b3bc55600dba)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:02.860969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a222f1-e1b4-4147-8ca9-86488f11bf55 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:02.909529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3a222f1-e1b4-4147-8ca9-86488f11bf55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff397d6b3f4a42bf8b68b3bc55600dba', 'request_id': 'req-c3a222f1-e1b4-4147-8ca9-86488f11bf55'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:22:02.915584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3a222f1-e1b4-4147-8ca9-86488f11bf55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff397d6b3f4a42bf8b68b3bc55600dba', 'request_id': 'req-c3a222f1-e1b4-4147-8ca9-86488f11bf55'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:22:02.922657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3a222f1-e1b4-4147-8ca9-86488f11bf55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff397d6b3f4a42bf8b68b3bc55600dba', 'request_id': 'req-c3a222f1-e1b4-4147-8ca9-86488f11bf55'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:22:02.924452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:22:02.939860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3a222f1-e1b4-4147-8ca9-86488f11bf55 admin admin] The token cache is being invalidate because project ff397d6b3f4a42bf8b68b3bc55600dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:22:02.940162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3a222f1-e1b4-4147-8ca9-86488f11bf55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:22:02.940851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3a222f1-e1b4-4147-8ca9-86488f11bf55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:22:02.942420 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1237/2464] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:22:02 2022] DELETE /identity/v3/projects/ff397d6b3f4a42bf8b68b3bc55600dba => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:22:02.948022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627de8e-2df2-444a-8b30-32de9f29d42b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:02.948568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627de8e-2df2-444a-8b30-32de9f29d42b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:02.948865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627de8e-2df2-444a-8b30-32de9f29d42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:02.978242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:22:03.012245 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b627de8e-2df2-444a-8b30-32de9f29d42b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:22:03.013563 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b627de8e-2df2-444a-8b30-32de9f29d42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:03.075052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1228/2465] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:22:03.083070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73996c0-a2d9-49fd-a66b-548745efcbc2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.083359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73996c0-a2d9-49fd-a66b-548745efcbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.083637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73996c0-a2d9-49fd-a66b-548745efcbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.107362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f73996c0-a2d9-49fd-a66b-548745efcbc2 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:22:03.108707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f73996c0-a2d9-49fd-a66b-548745efcbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:03.116720 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1238/2466] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:22:03.124297 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627de8e-2df2-444a-8b30-32de9f29d42b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:03.134720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627de8e-2df2-444a-8b30-32de9f29d42b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:03.140805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc5638-19fa-4407-baaf-065da5acae81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:03.141559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc5638-19fa-4407-baaf-065da5acae81 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.141834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc5638-19fa-4407-baaf-065da5acae81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.142098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc5638-19fa-4407-baaf-065da5acae81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.142736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc5638-19fa-4407-baaf-065da5acae81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:22:03.143029 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc5638-19fa-4407-baaf-065da5acae81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:03.144624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc5638-19fa-4407-baaf-065da5acae81 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:03.158884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1229/2467] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:22:03.163968 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73996c0-a2d9-49fd-a66b-548745efcbc2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:03.176772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73996c0-a2d9-49fd-a66b-548745efcbc2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:03.181888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b69a2-4c33-4942-9825-66295d9f60b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:03.182498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b69a2-4c33-4942-9825-66295d9f60b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.182778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b69a2-4c33-4942-9825-66295d9f60b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.183048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b69a2-4c33-4942-9825-66295d9f60b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.183686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b69a2-4c33-4942-9825-66295d9f60b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:03.185728 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b69a2-4c33-4942-9825-66295d9f60b0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:03.219848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1239/2468] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:22:03 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:22:03.224246 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc5638-19fa-4407-baaf-065da5acae81 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:03.235119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc5638-19fa-4407-baaf-065da5acae81 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:03.297185 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d2535-4331-442a-a27f-36e353fc99f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:03.298306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d2535-4331-442a-a27f-36e353fc99f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.298586 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d2535-4331-442a-a27f-36e353fc99f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.299721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d2535-4331-442a-a27f-36e353fc99f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.300173 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d2535-4331-442a-a27f-36e353fc99f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:03.302154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d2535-4331-442a-a27f-36e353fc99f6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:03.326386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1230/2469] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:22:03 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:22:03.330434 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b69a2-4c33-4942-9825-66295d9f60b0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:03.341262 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b69a2-4c33-4942-9825-66295d9f60b0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:03.346104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f46c6-a477-4a71-8ec8-8355dddd3eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:03.346729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f46c6-a477-4a71-8ec8-8355dddd3eff admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.347002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f46c6-a477-4a71-8ec8-8355dddd3eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.347274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f46c6-a477-4a71-8ec8-8355dddd3eff admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.347998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f46c6-a477-4a71-8ec8-8355dddd3eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:03.349658 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f46c6-a477-4a71-8ec8-8355dddd3eff admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:03.367303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1240/2470] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:22:03.368599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d2535-4331-442a-a27f-36e353fc99f6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:03.380164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d2535-4331-442a-a27f-36e353fc99f6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:03.386239 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9294118-8708-4c04-8fec-baf9294f9d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:03.386896 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9294118-8708-4c04-8fec-baf9294f9d6f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.387104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9294118-8708-4c04-8fec-baf9294f9d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.387384 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9294118-8708-4c04-8fec-baf9294f9d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.388030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9294118-8708-4c04-8fec-baf9294f9d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:03.389701 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9294118-8708-4c04-8fec-baf9294f9d6f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:03.400952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1231/2471] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:22:03.405372 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f46c6-a477-4a71-8ec8-8355dddd3eff admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:03.416334 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f46c6-a477-4a71-8ec8-8355dddd3eff admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:03.421003 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a2122-4aa3-42bd-bbf9-2d462ef605e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:03.421663 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a2122-4aa3-42bd-bbf9-2d462ef605e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.421940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a2122-4aa3-42bd-bbf9-2d462ef605e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.422207 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a2122-4aa3-42bd-bbf9-2d462ef605e4 admin admin] PATH_INFO: `/v3/projects/0ca0b63bd9224370b45e7cc5e9c68a27/users/e22472efcb9a454688744457cb45eabb/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.439508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a2122-4aa3-42bd-bbf9-2d462ef605e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca0b63bd9224370b45e7cc5e9c68a27, user_id=e22472efcb9a454688744457cb45eabb, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:03.442366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a2122-4aa3-42bd-bbf9-2d462ef605e4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:03.459145 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1241/2472] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:22:03 2022] PUT /identity/v3/projects/0ca0b63bd9224370b45e7cc5e9c68a27/users/e22472efcb9a454688744457cb45eabb/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:22:03.464740 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf632ca4-8515-4b7f-ba2a-023b1247ed61 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.465223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf632ca4-8515-4b7f-ba2a-023b1247ed61 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.465373 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf632ca4-8515-4b7f-ba2a-023b1247ed61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.486635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cf632ca4-8515-4b7f-ba2a-023b1247ed61 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:22:03.487913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cf632ca4-8515-4b7f-ba2a-023b1247ed61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:03.553451 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:22:03.584123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1232/2473] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:22:03.597760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a2122-4aa3-42bd-bbf9-2d462ef605e4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:03.599105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c4eda-73f5-4805-91c5-81be3970a352 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.599396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c4eda-73f5-4805-91c5-81be3970a352 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.599718 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c4eda-73f5-4805-91c5-81be3970a352 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.600843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3f0a2122-4aa3-42bd-bbf9-2d462ef605e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:03.601114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1233/2474] 10.222.0.79 () {54 vars in 921 bytes} [Tue Sep 6 21:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 21:22:03.606928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0f979-aae2-4f4c-8132-6daa0a247b15 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.607210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0f979-aae2-4f4c-8132-6daa0a247b15 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.607480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0f979-aae2-4f4c-8132-6daa0a247b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.642170 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-44c0f979-aae2-4f4c-8132-6daa0a247b15 None None] MFA Rules not processed for user `d5a45dec40844877a6b5b301fc44a110`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:22:03.643436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-44c0f979-aae2-4f4c-8132-6daa0a247b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:03.687572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a2122-4aa3-42bd-bbf9-2d462ef605e4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:03.694731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf134e-1bda-4d8c-8e66-fa645c549ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:03.695375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf134e-1bda-4d8c-8e66-fa645c549ffd service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.695656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf134e-1bda-4d8c-8e66-fa645c549ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.695933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf134e-1bda-4d8c-8e66-fa645c549ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.709679 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf134e-1bda-4d8c-8e66-fa645c549ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:03.711321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf134e-1bda-4d8c-8e66-fa645c549ffd service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:03.728494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1242/2475] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:22:03.729423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1234/2476] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Sep 6 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:22:03.743051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf134e-1bda-4d8c-8e66-fa645c549ffd service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:03.745808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-febf134e-1bda-4d8c-8e66-fa645c549ffd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:03.757912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf134e-1bda-4d8c-8e66-fa645c549ffd service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:03.763011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422df5d-1e35-4436-8c38-66714eeddadd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:03.764031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422df5d-1e35-4436-8c38-66714eeddadd service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.764213 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422df5d-1e35-4436-8c38-66714eeddadd service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.764500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422df5d-1e35-4436-8c38-66714eeddadd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.773349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422df5d-1e35-4436-8c38-66714eeddadd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:03.775181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422df5d-1e35-4436-8c38-66714eeddadd service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:03.785037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1243/2477] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:22:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:22:03.893745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7455a6b-8b38-49af-9e66-b8f2b5de3a3a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.894010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7455a6b-8b38-49af-9e66-b8f2b5de3a3a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.894268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7455a6b-8b38-49af-9e66-b8f2b5de3a3a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.896040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1235/2478] 10.222.0.79 () {54 vars in 921 bytes} [Tue Sep 6 21:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 21:22:03.903563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa8300-af80-4803-b520-b9bda2498203 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:03.903851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa8300-af80-4803-b520-b9bda2498203 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:03.904246 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa8300-af80-4803-b520-b9bda2498203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:03.945176 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-21aa8300-af80-4803-b520-b9bda2498203 None None] MFA Rules not processed for user `12d51454d20c46d7a5e6d963a2f987cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:22:03.946485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-21aa8300-af80-4803-b520-b9bda2498203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:04.009136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:22:04.039912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1244/2479] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Sep 6 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:22:04.051354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7455a6b-8b38-49af-9e66-b8f2b5de3a3a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:04.062635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7455a6b-8b38-49af-9e66-b8f2b5de3a3a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:04.066369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6514c79-2688-440a-9a36-2174c0216c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:04.067126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6514c79-2688-440a-9a36-2174c0216c2f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:04.067404 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6514c79-2688-440a-9a36-2174c0216c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:04.067676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6514c79-2688-440a-9a36-2174c0216c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:04.079547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6514c79-2688-440a-9a36-2174c0216c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:04.081425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6514c79-2688-440a-9a36-2174c0216c2f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:04.092510 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1236/2480] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:22:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:22:09.826184 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8139b6-600d-4eb5-a296-2ff54c7eaedf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:09.826740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8139b6-600d-4eb5-a296-2ff54c7eaedf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:09.826740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8139b6-600d-4eb5-a296-2ff54c7eaedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:09.865708 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6e8139b6-600d-4eb5-a296-2ff54c7eaedf None None] MFA Rules not processed for user `e22472efcb9a454688744457cb45eabb`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:22:09.867252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6e8139b6-600d-4eb5-a296-2ff54c7eaedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:09.942007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1245/2481] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:22:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:22:09.957566 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6514c79-2688-440a-9a36-2174c0216c2f service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:09.959057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b6514c79-2688-440a-9a36-2174c0216c2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:09.973631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6514c79-2688-440a-9a36-2174c0216c2f service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:09.979136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308cccc-074a-483e-bbcf-d1d12989c6b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:09.979783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308cccc-074a-483e-bbcf-d1d12989c6b3 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:09.980058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308cccc-074a-483e-bbcf-d1d12989c6b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:09.980326 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308cccc-074a-483e-bbcf-d1d12989c6b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:09.989184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308cccc-074a-483e-bbcf-d1d12989c6b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:09.990778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308cccc-074a-483e-bbcf-d1d12989c6b3 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:10.000524 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1237/2482] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:22:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:22:11.323940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e674d-1977-44c1-b182-3dec139c0ce9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:11.324502 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e674d-1977-44c1-b182-3dec139c0ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:11.324502 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e674d-1977-44c1-b182-3dec139c0ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:11.345016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7c4e674d-1977-44c1-b182-3dec139c0ce9 None None] MFA Rules not processed for user `e22472efcb9a454688744457cb45eabb`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:22:11.346846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7c4e674d-1977-44c1-b182-3dec139c0ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:11.358838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1246/2483] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:22:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:22:12.563046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308cccc-074a-483e-bbcf-d1d12989c6b3 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:22:12.564767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f308cccc-074a-483e-bbcf-d1d12989c6b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:12.654341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308cccc-074a-483e-bbcf-d1d12989c6b3 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:22:12.659947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7101d7-7c9e-45bf-b34e-5d4233a53f14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:22:12.660744 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7101d7-7c9e-45bf-b34e-5d4233a53f14 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:12.661091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7101d7-7c9e-45bf-b34e-5d4233a53f14 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:12.661427 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7101d7-7c9e-45bf-b34e-5d4233a53f14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:12.663496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fa7101d7-7c9e-45bf-b34e-5d4233a53f14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:12.677820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7101d7-7c9e-45bf-b34e-5d4233a53f14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:22:12.681454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7101d7-7c9e-45bf-b34e-5d4233a53f14 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:22:12.696590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1238/2484] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:22:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:22:21.338970 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=43 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:22:22.690413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=73 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:22:23.138926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b268f19-ec08-4867-a0ab-ad153406a5a2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:22:23.139529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b268f19-ec08-4867-a0ab-ad153406a5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:22:23.139529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b268f19-ec08-4867-a0ab-ad153406a5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:22:23.162411 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7b268f19-ec08-4867-a0ab-ad153406a5a2 None None] MFA Rules not processed for user `64033668b9fd4ba48eef09bdc3fdf2ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:22:23.164242 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7b268f19-ec08-4867-a0ab-ad153406a5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:22:23.227565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1247/2485] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:22:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:22:33.220585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=25 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:23:30.798409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98baef81-e909-490e-bb8e-a59dbc7d2be4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:23:30.799079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98baef81-e909-490e-bb8e-a59dbc7d2be4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:23:30.799079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98baef81-e909-490e-bb8e-a59dbc7d2be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:23:30.818992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-98baef81-e909-490e-bb8e-a59dbc7d2be4 None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:23:30.820184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-98baef81-e909-490e-bb8e-a59dbc7d2be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:23:30.882035 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1239/2486] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:23:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:23:30.931171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b268f19-ec08-4867-a0ab-ad153406a5a2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:23:30.932717 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7b268f19-ec08-4867-a0ab-ad153406a5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:23:31.004283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b268f19-ec08-4867-a0ab-ad153406a5a2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:23:31.009956 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba5931-4ba9-41ae-b406-3d6a4d3ac673 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:23:31.010569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba5931-4ba9-41ae-b406-3d6a4d3ac673 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:23:31.010846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba5931-4ba9-41ae-b406-3d6a4d3ac673 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:23:31.011117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba5931-4ba9-41ae-b406-3d6a4d3ac673 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:23:31.019621 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba5931-4ba9-41ae-b406-3d6a4d3ac673 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:23:31.021256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba5931-4ba9-41ae-b406-3d6a4d3ac673 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:23:31.029839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1248/2487] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:23:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:23:40.874645 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=25 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:23:41.026684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=32 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:23:42.778117 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98baef81-e909-490e-bb8e-a59dbc7d2be4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:23:42.800787 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98baef81-e909-490e-bb8e-a59dbc7d2be4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:23:42.805327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2538d-0e03-463d-a879-1361b95b1321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:23:42.805926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2538d-0e03-463d-a879-1361b95b1321 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:23:42.806200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2538d-0e03-463d-a879-1361b95b1321 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:23:42.806460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2538d-0e03-463d-a879-1361b95b1321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:23:42.808928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-52e2538d-0e03-463d-a879-1361b95b1321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:23:42.832872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2538d-0e03-463d-a879-1361b95b1321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:23:42.834690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2538d-0e03-463d-a879-1361b95b1321 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:23:42.846772 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1240/2488] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:23:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:23:44.050437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba5931-4ba9-41ae-b406-3d6a4d3ac673 service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:23:44.060851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba5931-4ba9-41ae-b406-3d6a4d3ac673 service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:23:44.064830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4874f19-6dea-4242-b1a7-acaeee818767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:23:44.065455 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4874f19-6dea-4242-b1a7-acaeee818767 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:23:44.065729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4874f19-6dea-4242-b1a7-acaeee818767 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:23:44.065994 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4874f19-6dea-4242-b1a7-acaeee818767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:23:44.067830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c4874f19-6dea-4242-b1a7-acaeee818767 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:23:44.082776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4874f19-6dea-4242-b1a7-acaeee818767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:23:44.084212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4874f19-6dea-4242-b1a7-acaeee818767 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:23:44.095693 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1249/2489] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:23:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:23:47.282163 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4bf58-7661-4df3-8351-55c4bc1f2cd4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:23:47.282756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4bf58-7661-4df3-8351-55c4bc1f2cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:23:47.282756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4bf58-7661-4df3-8351-55c4bc1f2cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:23:47.311556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4ba4bf58-7661-4df3-8351-55c4bc1f2cd4 None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:23:47.313203 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4ba4bf58-7661-4df3-8351-55c4bc1f2cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:23:47.323803 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1241/2490] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:23:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:23:47.336644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4874f19-6dea-4242-b1a7-acaeee818767 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:23:47.348768 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4874f19-6dea-4242-b1a7-acaeee818767 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:23:47.355028 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54ace7-8263-4fd0-8df5-0a621aa4cccd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:23:47.355636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54ace7-8263-4fd0-8df5-0a621aa4cccd service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:23:47.355913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54ace7-8263-4fd0-8df5-0a621aa4cccd service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:23:47.356178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54ace7-8263-4fd0-8df5-0a621aa4cccd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:23:47.364831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54ace7-8263-4fd0-8df5-0a621aa4cccd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:23:47.366435 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54ace7-8263-4fd0-8df5-0a621aa4cccd service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:23:47.374778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1250/2491] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:23:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:23:53.540820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4bf58-7661-4df3-8351-55c4bc1f2cd4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:23:53.555739 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4bf58-7661-4df3-8351-55c4bc1f2cd4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:23:53.563805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfff2e-dacd-47f9-97c2-d4cd0c28b942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:23:53.564410 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfff2e-dacd-47f9-97c2-d4cd0c28b942 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:23:53.564700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfff2e-dacd-47f9-97c2-d4cd0c28b942 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:23:53.564960 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfff2e-dacd-47f9-97c2-d4cd0c28b942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:23:53.567029 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-52bfff2e-dacd-47f9-97c2-d4cd0c28b942 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:23:53.670616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfff2e-dacd-47f9-97c2-d4cd0c28b942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:23:53.672881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfff2e-dacd-47f9-97c2-d4cd0c28b942 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:23:53.692683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1242/2492] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:23:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:23:57.371476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=12 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:24:02.141239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54ace7-8263-4fd0-8df5-0a621aa4cccd service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:02.152225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54ace7-8263-4fd0-8df5-0a621aa4cccd service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:02.156832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf317eb-3824-4535-b7c5-a3926df9b364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:02.157488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf317eb-3824-4535-b7c5-a3926df9b364 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:02.157759 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf317eb-3824-4535-b7c5-a3926df9b364 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:02.158019 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf317eb-3824-4535-b7c5-a3926df9b364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:02.160155 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5bf317eb-3824-4535-b7c5-a3926df9b364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:02.228398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf317eb-3824-4535-b7c5-a3926df9b364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:02.229922 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf317eb-3824-4535-b7c5-a3926df9b364 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:02.240747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1251/2493] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:24:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:24:03.689874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=54 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:24:07.173793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfff2e-dacd-47f9-97c2-d4cd0c28b942 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:07.176057 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-52bfff2e-dacd-47f9-97c2-d4cd0c28b942 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:07.313706 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfff2e-dacd-47f9-97c2-d4cd0c28b942 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:07.318460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f6f83-6997-4c74-84b3-60b58a8d52b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:07.319361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f6f83-6997-4c74-84b3-60b58a8d52b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:07.320360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f6f83-6997-4c74-84b3-60b58a8d52b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:07.320830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f6f83-6997-4c74-84b3-60b58a8d52b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:07.322956 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a45f6f83-6997-4c74-84b3-60b58a8d52b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:07.402106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f6f83-6997-4c74-84b3-60b58a8d52b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:07.403724 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f6f83-6997-4c74-84b3-60b58a8d52b1 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:07.414475 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1243/2494] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:24:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:24:09.626127 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcce123-770f-4caf-b9d9-179ba7240045 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:09.626713 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcce123-770f-4caf-b9d9-179ba7240045 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:09.626818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcce123-770f-4caf-b9d9-179ba7240045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:09.646373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5fcce123-770f-4caf-b9d9-179ba7240045 None None] MFA Rules not processed for user `e22472efcb9a454688744457cb45eabb`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:24:09.647706 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5fcce123-770f-4caf-b9d9-179ba7240045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:09.655173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1252/2495] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:24:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:24:09.925700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f6f83-6997-4c74-84b3-60b58a8d52b1 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:09.927106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a45f6f83-6997-4c74-84b3-60b58a8d52b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:09.944088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f6f83-6997-4c74-84b3-60b58a8d52b1 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:09.949894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d7d93-168f-4cef-a01b-d4ec2729c978 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:09.950523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d7d93-168f-4cef-a01b-d4ec2729c978 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:09.950790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d7d93-168f-4cef-a01b-d4ec2729c978 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:09.951044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d7d93-168f-4cef-a01b-d4ec2729c978 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:09.952850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0c5d7d93-168f-4cef-a01b-d4ec2729c978 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:09.987729 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d7d93-168f-4cef-a01b-d4ec2729c978 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:09.989393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d7d93-168f-4cef-a01b-d4ec2729c978 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:10.008642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1244/2496] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:24:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:24:13.717204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f6b46-7583-483a-bec4-e8437fa3bad3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:13.717786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f6b46-7583-483a-bec4-e8437fa3bad3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:13.717786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f6b46-7583-483a-bec4-e8437fa3bad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:13.772466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fe2f6b46-7583-483a-bec4-e8437fa3bad3 None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:24:13.774051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fe2f6b46-7583-483a-bec4-e8437fa3bad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:13.793558 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1253/2497] 10.222.0.79 () {58 vars in 967 bytes} [Tue Sep 6 21:24:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:24:13.807348 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d7d93-168f-4cef-a01b-d4ec2729c978 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:13.828536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d7d93-168f-4cef-a01b-d4ec2729c978 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:13.833502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd2520-4427-4eca-a87e-1a75dfe5ea70 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:13.834105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd2520-4427-4eca-a87e-1a75dfe5ea70 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:13.834383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd2520-4427-4eca-a87e-1a75dfe5ea70 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:13.834652 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd2520-4427-4eca-a87e-1a75dfe5ea70 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:13.843529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd2520-4427-4eca-a87e-1a75dfe5ea70 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:13.845229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd2520-4427-4eca-a87e-1a75dfe5ea70 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:13.855472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1245/2498] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:24:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:24:15.707733 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f6b46-7583-483a-bec4-e8437fa3bad3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:15.718979 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f6b46-7583-483a-bec4-e8437fa3bad3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:15.722844 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc451c8-0548-483a-9083-3d46f8ad1ab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:15.723472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc451c8-0548-483a-9083-3d46f8ad1ab8 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:15.723743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc451c8-0548-483a-9083-3d46f8ad1ab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:15.724395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc451c8-0548-483a-9083-3d46f8ad1ab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:15.726448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8bc451c8-0548-483a-9083-3d46f8ad1ab8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:15.752069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc451c8-0548-483a-9083-3d46f8ad1ab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:15.753851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc451c8-0548-483a-9083-3d46f8ad1ab8 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:15.763696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1254/2499] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:24:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:24:16.576921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd2520-4427-4eca-a87e-1a75dfe5ea70 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:16.594242 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd2520-4427-4eca-a87e-1a75dfe5ea70 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:16.598421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46acd2-f5b0-4a71-a6f2-f97b7cc8e2d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:16.599417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46acd2-f5b0-4a71-a6f2-f97b7cc8e2d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:16.599698 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46acd2-f5b0-4a71-a6f2-f97b7cc8e2d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:16.599969 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46acd2-f5b0-4a71-a6f2-f97b7cc8e2d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:16.601975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1c46acd2-f5b0-4a71-a6f2-f97b7cc8e2d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:16.615136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46acd2-f5b0-4a71-a6f2-f97b7cc8e2d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:16.616585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46acd2-f5b0-4a71-a6f2-f97b7cc8e2d9 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:16.627851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1246/2500] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:24:19.624503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e4c1c-11d1-4cd2-a3fb-6050f1932369 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:19.624972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e4c1c-11d1-4cd2-a3fb-6050f1932369 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:19.624972 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e4c1c-11d1-4cd2-a3fb-6050f1932369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:19.647574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f42e4c1c-11d1-4cd2-a3fb-6050f1932369 None None] MFA Rules not processed for user `6d0c1c510b0a4fa39927f7f92352d034`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:24:19.648846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f42e4c1c-11d1-4cd2-a3fb-6050f1932369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:19.656384 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1255/2501] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:24:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:24:20.696716 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46acd2-f5b0-4a71-a6f2-f97b7cc8e2d9 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:20.698683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1c46acd2-f5b0-4a71-a6f2-f97b7cc8e2d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:20.717110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46acd2-f5b0-4a71-a6f2-f97b7cc8e2d9 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:20.729626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e30ce-00f7-4cc5-9794-e82ba07ce022 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:20.734834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e30ce-00f7-4cc5-9794-e82ba07ce022 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:20.735292 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e30ce-00f7-4cc5-9794-e82ba07ce022 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:20.735733 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e30ce-00f7-4cc5-9794-e82ba07ce022 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:20.738472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-eb2e30ce-00f7-4cc5-9794-e82ba07ce022 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:20.755249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e30ce-00f7-4cc5-9794-e82ba07ce022 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:20.757850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e30ce-00f7-4cc5-9794-e82ba07ce022 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:20.769983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1247/2502] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:24:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:24:24.389294 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e4c1c-11d1-4cd2-a3fb-6050f1932369 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:24.401380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e30ce-00f7-4cc5-9794-e82ba07ce022 service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:24.403114 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-eb2e30ce-00f7-4cc5-9794-e82ba07ce022 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:24.422836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:24:24.428029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e4c1c-11d1-4cd2-a3fb-6050f1932369 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:24.430219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e30ce-00f7-4cc5-9794-e82ba07ce022 service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:24.434146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c809b-218b-4d98-8647-6a6aa6f33964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:24.434807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c809b-218b-4d98-8647-6a6aa6f33964 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:24.435088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c809b-218b-4d98-8647-6a6aa6f33964 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:24.435356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c809b-218b-4d98-8647-6a6aa6f33964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:24.437175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-266c809b-218b-4d98-8647-6a6aa6f33964 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:24.445866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:24.447433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:24.447806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:24.448227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] PATH_INFO: `/v3/users/b9d62ee769bb4097b02703cca87ee6f4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:24.449818 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d62ee769bb4097b02703cca87ee6f4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:24.451425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c809b-218b-4d98-8647-6a6aa6f33964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:24.451753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:24.453329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c809b-218b-4d98-8647-6a6aa6f33964 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:24.470474 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1256/2503] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:24:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:24:24.536695 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d62ee769bb4097b02703cca87ee6f4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:24.537576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d62ee769bb4097b02703cca87ee6f4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:24.545448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d62ee769bb4097b02703cca87ee6f4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:24.559278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d62ee769bb4097b02703cca87ee6f4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:24.560032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d62ee769bb4097b02703cca87ee6f4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:24.574768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1248/2504] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:24:24 2022] DELETE /identity/v3/users/b9d62ee769bb4097b02703cca87ee6f4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:24:24.758847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c809b-218b-4d98-8647-6a6aa6f33964 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:24.772277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c809b-218b-4d98-8647-6a6aa6f33964 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:24.854869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:24.855673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:24.856231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:24.857749 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] PATH_INFO: `/v3/projects/fd90c9c3a5554874b6b1d2e5d32d11f9` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:24.859276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd90c9c3a5554874b6b1d2e5d32d11f9)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:24.861534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:24.873753 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:24:24.920780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd90c9c3a5554874b6b1d2e5d32d11f9', 'request_id': 'req-035cdeff-37df-4f07-9e7a-5ad8267a8c13'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:24.927625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd90c9c3a5554874b6b1d2e5d32d11f9', 'request_id': 'req-035cdeff-37df-4f07-9e7a-5ad8267a8c13'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:24.935244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd90c9c3a5554874b6b1d2e5d32d11f9', 'request_id': 'req-035cdeff-37df-4f07-9e7a-5ad8267a8c13'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:24.953652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] The token cache is being invalidate because project fd90c9c3a5554874b6b1d2e5d32d11f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:24:24.954194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:24.955040 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:24.956912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1257/2505] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:24:24 2022] DELETE /identity/v3/projects/fd90c9c3a5554874b6b1d2e5d32d11f9 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:24:24.961053 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:24.962770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:24.987055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ea8f8-9d5e-4dad-a944-7791b70fb75a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:25.067209 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:25.067921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:25.068208 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:25.068473 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] PATH_INFO: `/v3/users/d90fa5e88bd543a68cd5d3c70b0b6b2c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:25.070119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d90fa5e88bd543a68cd5d3c70b0b6b2c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:25.071949 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:25.155144 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd90fa5e88bd543a68cd5d3c70b0b6b2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:25.155892 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd90fa5e88bd543a68cd5d3c70b0b6b2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:25.164279 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd90fa5e88bd543a68cd5d3c70b0b6b2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:25.179454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd90fa5e88bd543a68cd5d3c70b0b6b2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:25.180390 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd90fa5e88bd543a68cd5d3c70b0b6b2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:25.185022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:24:25.203263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1249/2506] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:24:24 2022] DELETE /identity/v3/users/d90fa5e88bd543a68cd5d3c70b0b6b2c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:24:25.604409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:25.606923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:25.620604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cdeff-37df-4f07-9e7a-5ad8267a8c13 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:25.690838 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:25.691479 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:25.691756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:25.692055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] PATH_INFO: `/v3/projects/2bb19cbf237f4cd7bb3fb6ea582cc1bb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:25.693197 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb19cbf237f4cd7bb3fb6ea582cc1bb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:25.695772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:25.745000 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb19cbf237f4cd7bb3fb6ea582cc1bb', 'request_id': 'req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:25.751162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb19cbf237f4cd7bb3fb6ea582cc1bb', 'request_id': 'req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:25.757018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb19cbf237f4cd7bb3fb6ea582cc1bb', 'request_id': 'req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:25.770067 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] The token cache is being invalidate because project 2bb19cbf237f4cd7bb3fb6ea582cc1bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:24:25.770370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:25.771093 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:25.772853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1258/2507] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:24:25 2022] DELETE /identity/v3/projects/2bb19cbf237f4cd7bb3fb6ea582cc1bb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:24:35.202251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:24:35.766981 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=71,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:24:44.036446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:44.038522 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:44.116482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b683117-a33e-4b92-ada8-f6ceb4f484cc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:44.174874 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300b77d-dc0d-4e71-94a5-1a2add0b03d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:44.175549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300b77d-dc0d-4e71-94a5-1a2add0b03d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:44.175862 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300b77d-dc0d-4e71-94a5-1a2add0b03d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:44.176127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300b77d-dc0d-4e71-94a5-1a2add0b03d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:44.177940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0300b77d-dc0d-4e71-94a5-1a2add0b03d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:44.241115 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300b77d-dc0d-4e71-94a5-1a2add0b03d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:44.242557 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300b77d-dc0d-4e71-94a5-1a2add0b03d4 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:44.252050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1250/2508] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:24:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:24:52.991403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:52.993483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:53.009755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa60d19-cd61-4641-9fc9-bacf90ab4f38 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:53.014344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5e078-c6af-49ce-8014-e86f8457c17b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:53.014934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5e078-c6af-49ce-8014-e86f8457c17b service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:53.015209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5e078-c6af-49ce-8014-e86f8457c17b service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:53.015505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5e078-c6af-49ce-8014-e86f8457c17b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:53.017459 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1ff5e078-c6af-49ce-8014-e86f8457c17b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:53.092918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5e078-c6af-49ce-8014-e86f8457c17b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:53.094366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5e078-c6af-49ce-8014-e86f8457c17b service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:53.107100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1259/2509] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:24:54.247799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=70 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:24:56.459609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300b77d-dc0d-4e71-94a5-1a2add0b03d4 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:56.461202 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0300b77d-dc0d-4e71-94a5-1a2add0b03d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:56.474997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300b77d-dc0d-4e71-94a5-1a2add0b03d4 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:56.479953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615432e7-3651-41c6-be3c-6b2fc83a01d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:56.480591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615432e7-3651-41c6-be3c-6b2fc83a01d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:56.480894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615432e7-3651-41c6-be3c-6b2fc83a01d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:56.481163 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615432e7-3651-41c6-be3c-6b2fc83a01d6 admin admin] PATH_INFO: `/v3/users/64033668b9fd4ba48eef09bdc3fdf2ee` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:56.482298 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615432e7-3651-41c6-be3c-6b2fc83a01d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64033668b9fd4ba48eef09bdc3fdf2ee)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:56.484232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615432e7-3651-41c6-be3c-6b2fc83a01d6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:56.536069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-615432e7-3651-41c6-be3c-6b2fc83a01d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64033668b9fd4ba48eef09bdc3fdf2ee'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:56.536881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-615432e7-3651-41c6-be3c-6b2fc83a01d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64033668b9fd4ba48eef09bdc3fdf2ee'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:56.545866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-615432e7-3651-41c6-be3c-6b2fc83a01d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64033668b9fd4ba48eef09bdc3fdf2ee'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:56.561263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-615432e7-3651-41c6-be3c-6b2fc83a01d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64033668b9fd4ba48eef09bdc3fdf2ee'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:56.562331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-615432e7-3651-41c6-be3c-6b2fc83a01d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64033668b9fd4ba48eef09bdc3fdf2ee'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:56.579745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1251/2510] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:24:56 2022] DELETE /identity/v3/users/64033668b9fd4ba48eef09bdc3fdf2ee => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:24:56.778043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5e078-c6af-49ce-8014-e86f8457c17b service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:56.789520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5e078-c6af-49ce-8014-e86f8457c17b service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:56.860807 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6ecab-74ed-4cf9-8219-c1735b707f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:56.861558 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6ecab-74ed-4cf9-8219-c1735b707f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:56.862042 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6ecab-74ed-4cf9-8219-c1735b707f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:56.862495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6ecab-74ed-4cf9-8219-c1735b707f74 admin admin] PATH_INFO: `/v3/projects/cabcd82424984a0ba003e5324b505e54` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:56.863843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6ecab-74ed-4cf9-8219-c1735b707f74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cabcd82424984a0ba003e5324b505e54)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:56.865852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6ecab-74ed-4cf9-8219-c1735b707f74 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:56.924193 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8e6ecab-74ed-4cf9-8219-c1735b707f74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cabcd82424984a0ba003e5324b505e54', 'request_id': 'req-d8e6ecab-74ed-4cf9-8219-c1735b707f74'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:56.931395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8e6ecab-74ed-4cf9-8219-c1735b707f74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cabcd82424984a0ba003e5324b505e54', 'request_id': 'req-d8e6ecab-74ed-4cf9-8219-c1735b707f74'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:56.938087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8e6ecab-74ed-4cf9-8219-c1735b707f74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cabcd82424984a0ba003e5324b505e54', 'request_id': 'req-d8e6ecab-74ed-4cf9-8219-c1735b707f74'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:56.956272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8e6ecab-74ed-4cf9-8219-c1735b707f74 admin admin] The token cache is being invalidate because project cabcd82424984a0ba003e5324b505e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:24:56.956565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8e6ecab-74ed-4cf9-8219-c1735b707f74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:56.957246 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8e6ecab-74ed-4cf9-8219-c1735b707f74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:24:56.958761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1260/2511] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:24:56 2022] DELETE /identity/v3/projects/cabcd82424984a0ba003e5324b505e54 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:24:56.964454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73b758-85b9-44a2-93cb-4c35a3fbd66d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:56.964758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73b758-85b9-44a2-93cb-4c35a3fbd66d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:56.965032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73b758-85b9-44a2-93cb-4c35a3fbd66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:56.985106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7c73b758-85b9-44a2-93cb-4c35a3fbd66d None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:24:56.986313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7c73b758-85b9-44a2-93cb-4c35a3fbd66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:57.055474 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1252/2512] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:24:57.063827 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed1358-4d38-4f71-9ef7-d55666471dc0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:57.064108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed1358-4d38-4f71-9ef7-d55666471dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:57.064375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed1358-4d38-4f71-9ef7-d55666471dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:57.085188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-83ed1358-4d38-4f71-9ef7-d55666471dc0 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:24:57.086424 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-83ed1358-4d38-4f71-9ef7-d55666471dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:57.096700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1261/2513] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:24:57.101761 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73b758-85b9-44a2-93cb-4c35a3fbd66d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:57.112018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73b758-85b9-44a2-93cb-4c35a3fbd66d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:57.117249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba6d87-85a1-48a1-a3b0-3caf626eb65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:57.117894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba6d87-85a1-48a1-a3b0-3caf626eb65b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:57.118131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba6d87-85a1-48a1-a3b0-3caf626eb65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:57.118419 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba6d87-85a1-48a1-a3b0-3caf626eb65b admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:57.119077 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba6d87-85a1-48a1-a3b0-3caf626eb65b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:24:57.119409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba6d87-85a1-48a1-a3b0-3caf626eb65b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:57.121110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba6d87-85a1-48a1-a3b0-3caf626eb65b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:57.133970 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1253/2514] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:24:57.138583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed1358-4d38-4f71-9ef7-d55666471dc0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:57.148183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed1358-4d38-4f71-9ef7-d55666471dc0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:57.153397 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04daa8-ea36-42e0-89e6-8b22deb6f843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:57.154015 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04daa8-ea36-42e0-89e6-8b22deb6f843 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:57.154286 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04daa8-ea36-42e0-89e6-8b22deb6f843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:57.154557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04daa8-ea36-42e0-89e6-8b22deb6f843 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:57.155195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04daa8-ea36-42e0-89e6-8b22deb6f843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:57.157215 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04daa8-ea36-42e0-89e6-8b22deb6f843 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:57.176218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:24:57.204460 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1262/2515] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:24:57 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:24:57.208933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba6d87-85a1-48a1-a3b0-3caf626eb65b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:57.219688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba6d87-85a1-48a1-a3b0-3caf626eb65b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:57.286195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e8232-09dc-4cb9-9236-004e4827ec33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:57.286971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e8232-09dc-4cb9-9236-004e4827ec33 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:57.287359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e8232-09dc-4cb9-9236-004e4827ec33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:57.287623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e8232-09dc-4cb9-9236-004e4827ec33 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:57.288490 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e8232-09dc-4cb9-9236-004e4827ec33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:57.291521 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e8232-09dc-4cb9-9236-004e4827ec33 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:57.301449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:24:57.320058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1254/2516] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:24:57 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:24:57.324408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04daa8-ea36-42e0-89e6-8b22deb6f843 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:57.335359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04daa8-ea36-42e0-89e6-8b22deb6f843 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:57.340687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9062261-e5c7-4959-8d12-fd7f724cae08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:57.341304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9062261-e5c7-4959-8d12-fd7f724cae08 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:57.341579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9062261-e5c7-4959-8d12-fd7f724cae08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:57.341846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9062261-e5c7-4959-8d12-fd7f724cae08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:57.342493 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9062261-e5c7-4959-8d12-fd7f724cae08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:57.344143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9062261-e5c7-4959-8d12-fd7f724cae08 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:57.354897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1263/2517] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:24:57.359208 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e8232-09dc-4cb9-9236-004e4827ec33 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:57.371295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e8232-09dc-4cb9-9236-004e4827ec33 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:57.377508 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89969fe-ba24-4d2e-b7f1-bf55a6fcd188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:57.379085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89969fe-ba24-4d2e-b7f1-bf55a6fcd188 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:57.380249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89969fe-ba24-4d2e-b7f1-bf55a6fcd188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:57.381532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89969fe-ba24-4d2e-b7f1-bf55a6fcd188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:57.386477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89969fe-ba24-4d2e-b7f1-bf55a6fcd188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:57.390902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89969fe-ba24-4d2e-b7f1-bf55a6fcd188 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:57.404360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1255/2518] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:24:57.408946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9062261-e5c7-4959-8d12-fd7f724cae08 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:57.420926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9062261-e5c7-4959-8d12-fd7f724cae08 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:57.426689 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b436e5e-8cef-40c7-93ef-d8ee58d9b746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:57.427356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b436e5e-8cef-40c7-93ef-d8ee58d9b746 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:57.427631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b436e5e-8cef-40c7-93ef-d8ee58d9b746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:57.427898 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b436e5e-8cef-40c7-93ef-d8ee58d9b746 admin admin] PATH_INFO: `/v3/projects/3df38e60a61948d5af82e97fa73da40a/users/c747c20a72c3455f86321af53cf803ba/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:57.446707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b436e5e-8cef-40c7-93ef-d8ee58d9b746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df38e60a61948d5af82e97fa73da40a, user_id=c747c20a72c3455f86321af53cf803ba, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:57.451823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b436e5e-8cef-40c7-93ef-d8ee58d9b746 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:57.469677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1264/2519] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:24:57 2022] PUT /identity/v3/projects/3df38e60a61948d5af82e97fa73da40a/users/c747c20a72c3455f86321af53cf803ba/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:24:57.480516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790265b9-fe97-4c93-9b19-df998fd3d69b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:57.480985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790265b9-fe97-4c93-9b19-df998fd3d69b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:57.481093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790265b9-fe97-4c93-9b19-df998fd3d69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:57.511238 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-790265b9-fe97-4c93-9b19-df998fd3d69b None None] MFA Rules not processed for user `c747c20a72c3455f86321af53cf803ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:24:57.512481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-790265b9-fe97-4c93-9b19-df998fd3d69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:57.584360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1256/2520] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:24:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:24:57.589287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b436e5e-8cef-40c7-93ef-d8ee58d9b746 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:57.652886 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b436e5e-8cef-40c7-93ef-d8ee58d9b746 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:57.657585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b010b-e717-49ae-9907-65db1ec97919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:57.658208 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b010b-e717-49ae-9907-65db1ec97919 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:57.658489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b010b-e717-49ae-9907-65db1ec97919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:57.658771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b010b-e717-49ae-9907-65db1ec97919 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:57.659447 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b010b-e717-49ae-9907-65db1ec97919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:57.661428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b010b-e717-49ae-9907-65db1ec97919 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:57.690200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1265/2521] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:24:57 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:24:57.694248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790265b9-fe97-4c93-9b19-df998fd3d69b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:57.705338 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790265b9-fe97-4c93-9b19-df998fd3d69b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:57.765512 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0766705-4002-4fdd-acba-446243ec75e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:57.766168 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0766705-4002-4fdd-acba-446243ec75e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:57.766447 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0766705-4002-4fdd-acba-446243ec75e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:57.766717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0766705-4002-4fdd-acba-446243ec75e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:57.767372 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0766705-4002-4fdd-acba-446243ec75e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:57.769367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0766705-4002-4fdd-acba-446243ec75e7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:57.792954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1257/2522] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:24:57 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:24:57.800106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b010b-e717-49ae-9907-65db1ec97919 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:57.811774 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b010b-e717-49ae-9907-65db1ec97919 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:57.816865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb0aee-6095-450b-ae5c-54385074e589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:57.817500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb0aee-6095-450b-ae5c-54385074e589 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:57.817779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb0aee-6095-450b-ae5c-54385074e589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:57.818052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb0aee-6095-450b-ae5c-54385074e589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:57.818705 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb0aee-6095-450b-ae5c-54385074e589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:57.820389 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb0aee-6095-450b-ae5c-54385074e589 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:57.832182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1266/2523] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:24:57.837179 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0766705-4002-4fdd-acba-446243ec75e7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:57.847169 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0766705-4002-4fdd-acba-446243ec75e7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:57.852326 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663ba3a-9a75-4897-8169-6675278b3385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:57.853562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663ba3a-9a75-4897-8169-6675278b3385 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:57.853562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663ba3a-9a75-4897-8169-6675278b3385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:57.853562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663ba3a-9a75-4897-8169-6675278b3385 admin admin] PATH_INFO: `/v3/projects/27308b6c40a24ca684e390dc72d6b955/users/975cde5b103148d896e1b5f6ed4f7b9d/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:57.870491 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663ba3a-9a75-4897-8169-6675278b3385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27308b6c40a24ca684e390dc72d6b955, user_id=975cde5b103148d896e1b5f6ed4f7b9d, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:57.873211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663ba3a-9a75-4897-8169-6675278b3385 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:57.895957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1258/2524] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:24:57 2022] PUT /identity/v3/projects/27308b6c40a24ca684e390dc72d6b955/users/975cde5b103148d896e1b5f6ed4f7b9d/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:24:57.900209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb0aee-6095-450b-ae5c-54385074e589 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:57.911168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb0aee-6095-450b-ae5c-54385074e589 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:57.985117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b2a48-5cac-4e03-b0d2-6f035f755f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:57.985776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b2a48-5cac-4e03-b0d2-6f035f755f50 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:57.986059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b2a48-5cac-4e03-b0d2-6f035f755f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:57.986333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b2a48-5cac-4e03-b0d2-6f035f755f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:57.986983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b2a48-5cac-4e03-b0d2-6f035f755f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:57.988635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b2a48-5cac-4e03-b0d2-6f035f755f50 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:58.000612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1267/2525] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:24:58.005025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663ba3a-9a75-4897-8169-6675278b3385 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:58.016247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663ba3a-9a75-4897-8169-6675278b3385 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:58.021813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a535a6-6b4b-428f-b7ab-ac231f391301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:58.022479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a535a6-6b4b-428f-b7ab-ac231f391301 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:58.022757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a535a6-6b4b-428f-b7ab-ac231f391301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:58.023195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a535a6-6b4b-428f-b7ab-ac231f391301 admin admin] PATH_INFO: `/v3/projects/27308b6c40a24ca684e390dc72d6b955/users/975cde5b103148d896e1b5f6ed4f7b9d/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:58.025429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a535a6-6b4b-428f-b7ab-ac231f391301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27308b6c40a24ca684e390dc72d6b955, user_id=975cde5b103148d896e1b5f6ed4f7b9d, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:58.028171 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a535a6-6b4b-428f-b7ab-ac231f391301 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:58.043663 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1259/2526] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:24:58 2022] PUT /identity/v3/projects/27308b6c40a24ca684e390dc72d6b955/users/975cde5b103148d896e1b5f6ed4f7b9d/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:24:58.055823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc1eb8-7125-4773-bf72-85c4654c9420 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:58.056087 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc1eb8-7125-4773-bf72-85c4654c9420 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:58.056376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc1eb8-7125-4773-bf72-85c4654c9420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:58.094878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a4cc1eb8-7125-4773-bf72-85c4654c9420 None None] MFA Rules not processed for user `975cde5b103148d896e1b5f6ed4f7b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:24:58.096162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a4cc1eb8-7125-4773-bf72-85c4654c9420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:58.106739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:24:58.195683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1268/2527] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:24:58.213531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a535a6-6b4b-428f-b7ab-ac231f391301 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:58.215712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b2a535a6-6b4b-428f-b7ab-ac231f391301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:24:58.244317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:24:58.284800 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a535a6-6b4b-428f-b7ab-ac231f391301 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:58.289249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ba695-9bfe-46c3-8a66-974d35540051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:58.289854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ba695-9bfe-46c3-8a66-974d35540051 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:58.290124 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ba695-9bfe-46c3-8a66-974d35540051 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:58.290388 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ba695-9bfe-46c3-8a66-974d35540051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:58.301424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ba695-9bfe-46c3-8a66-974d35540051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:58.302898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ba695-9bfe-46c3-8a66-974d35540051 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:58.312930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1260/2528] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:24:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:24:58.395847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc1eb8-7125-4773-bf72-85c4654c9420 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:24:58.410836 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc1eb8-7125-4773-bf72-85c4654c9420 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:24:58.415365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc87a1-a399-4197-a5c1-56fe33a8c697 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:24:58.415997 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc87a1-a399-4197-a5c1-56fe33a8c697 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:24:58.416273 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc87a1-a399-4197-a5c1-56fe33a8c697 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:24:58.416548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc87a1-a399-4197-a5c1-56fe33a8c697 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:24:58.479672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc87a1-a399-4197-a5c1-56fe33a8c697 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:24:58.481096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc87a1-a399-4197-a5c1-56fe33a8c697 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:24:58.491064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1269/2529] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:24:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:25:08.308291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=17 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:08.486541 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=52 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:09.589216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ba695-9bfe-46c3-8a66-974d35540051 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:09.653994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ba695-9bfe-46c3-8a66-974d35540051 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:09.659192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:09.660084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:09.660468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:09.660901 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] PATH_INFO: `/v3/users/c747c20a72c3455f86321af53cf803ba` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:09.662131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c747c20a72c3455f86321af53cf803ba)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:09.664350 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:09.718577 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c747c20a72c3455f86321af53cf803ba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:09.719238 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c747c20a72c3455f86321af53cf803ba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:09.727739 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c747c20a72c3455f86321af53cf803ba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:09.744284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c747c20a72c3455f86321af53cf803ba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:09.745165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c747c20a72c3455f86321af53cf803ba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:09.760125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1261/2530] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:09 2022] DELETE /identity/v3/users/c747c20a72c3455f86321af53cf803ba => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:09.765144 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27973050-5c89-499c-b591-83432b969997 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:09.765677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27973050-5c89-499c-b591-83432b969997 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:09.765677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27973050-5c89-499c-b591-83432b969997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:09.786673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-27973050-5c89-499c-b591-83432b969997 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:09.787886 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-27973050-5c89-499c-b591-83432b969997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:09.853750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1270/2531] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:09.863964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:09.865595 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:09.930871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907b50d-4cf9-415e-ae89-750dba58b3d1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:09.934893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4e79e-7566-4f62-a7cf-8de8100399a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:09.935511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4e79e-7566-4f62-a7cf-8de8100399a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:09.935787 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4e79e-7566-4f62-a7cf-8de8100399a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:09.936055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4e79e-7566-4f62-a7cf-8de8100399a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:09.945870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4e79e-7566-4f62-a7cf-8de8100399a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:09.947326 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4e79e-7566-4f62-a7cf-8de8100399a0 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:09.957300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1262/2532] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:10.245881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27973050-5c89-499c-b591-83432b969997 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:10.247324 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-27973050-5c89-499c-b591-83432b969997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:10.259828 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27973050-5c89-499c-b591-83432b969997 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:10.265354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:10.266007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:10.266286 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:10.266589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] PATH_INFO: `/v3/projects/3df38e60a61948d5af82e97fa73da40a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:10.267850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df38e60a61948d5af82e97fa73da40a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:10.269932 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:10.322500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df38e60a61948d5af82e97fa73da40a', 'request_id': 'req-40a80868-3df5-463d-bae5-27235c6a898b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:10.328878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df38e60a61948d5af82e97fa73da40a', 'request_id': 'req-40a80868-3df5-463d-bae5-27235c6a898b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:10.335106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df38e60a61948d5af82e97fa73da40a', 'request_id': 'req-40a80868-3df5-463d-bae5-27235c6a898b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:10.350704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] The token cache is being invalidate because project 3df38e60a61948d5af82e97fa73da40a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:10.351052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:10.351739 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:10.353374 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1271/2533] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:10 2022] DELETE /identity/v3/projects/3df38e60a61948d5af82e97fa73da40a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:10.357669 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4e79e-7566-4f62-a7cf-8de8100399a0 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:10.368077 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4e79e-7566-4f62-a7cf-8de8100399a0 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:10.388335 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:10.430579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192fc22-b714-484c-84ed-3f78bccf5811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:10.431219 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192fc22-b714-484c-84ed-3f78bccf5811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:10.431498 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192fc22-b714-484c-84ed-3f78bccf5811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:10.431770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192fc22-b714-484c-84ed-3f78bccf5811 admin admin] PATH_INFO: `/v3/users/975cde5b103148d896e1b5f6ed4f7b9d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:10.432974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192fc22-b714-484c-84ed-3f78bccf5811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=975cde5b103148d896e1b5f6ed4f7b9d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:10.434921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192fc22-b714-484c-84ed-3f78bccf5811 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:10.489885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e192fc22-b714-484c-84ed-3f78bccf5811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '975cde5b103148d896e1b5f6ed4f7b9d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:10.490917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e192fc22-b714-484c-84ed-3f78bccf5811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '975cde5b103148d896e1b5f6ed4f7b9d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:10.498260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e192fc22-b714-484c-84ed-3f78bccf5811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '975cde5b103148d896e1b5f6ed4f7b9d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:10.510907 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e192fc22-b714-484c-84ed-3f78bccf5811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '975cde5b103148d896e1b5f6ed4f7b9d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:10.511623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e192fc22-b714-484c-84ed-3f78bccf5811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '975cde5b103148d896e1b5f6ed4f7b9d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:10.528196 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1263/2534] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:10 2022] DELETE /identity/v3/users/975cde5b103148d896e1b5f6ed4f7b9d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:25:10.832100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:10.833976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:10.848552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a80868-3df5-463d-bae5-27235c6a898b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:10.907385 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a1786-034d-48f5-8f7b-6311e5cd909e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:10.908025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a1786-034d-48f5-8f7b-6311e5cd909e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:10.908301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a1786-034d-48f5-8f7b-6311e5cd909e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:10.908568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a1786-034d-48f5-8f7b-6311e5cd909e admin admin] PATH_INFO: `/v3/projects/27308b6c40a24ca684e390dc72d6b955` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:10.909712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a1786-034d-48f5-8f7b-6311e5cd909e admin admin] RBAC: Authorizing `identity:delete_project(project_id=27308b6c40a24ca684e390dc72d6b955)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:10.911668 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a1786-034d-48f5-8f7b-6311e5cd909e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:10.944548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:10.965122 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-411a1786-034d-48f5-8f7b-6311e5cd909e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27308b6c40a24ca684e390dc72d6b955', 'request_id': 'req-411a1786-034d-48f5-8f7b-6311e5cd909e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:10.972420 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-411a1786-034d-48f5-8f7b-6311e5cd909e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27308b6c40a24ca684e390dc72d6b955', 'request_id': 'req-411a1786-034d-48f5-8f7b-6311e5cd909e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:10.979097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-411a1786-034d-48f5-8f7b-6311e5cd909e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27308b6c40a24ca684e390dc72d6b955', 'request_id': 'req-411a1786-034d-48f5-8f7b-6311e5cd909e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:11.002989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-411a1786-034d-48f5-8f7b-6311e5cd909e admin admin] The token cache is being invalidate because project 27308b6c40a24ca684e390dc72d6b955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:11.003590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-411a1786-034d-48f5-8f7b-6311e5cd909e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:11.005116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-411a1786-034d-48f5-8f7b-6311e5cd909e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:11.007700 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1272/2535] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:10 2022] DELETE /identity/v3/projects/27308b6c40a24ca684e390dc72d6b955 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:11.014141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1b605-ce7f-4c55-9695-ffd74a78ddcc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:11.014711 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1b605-ce7f-4c55-9695-ffd74a78ddcc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:11.014711 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1b605-ce7f-4c55-9695-ffd74a78ddcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:11.039930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6ee1b605-ce7f-4c55-9695-ffd74a78ddcc None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:11.042940 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6ee1b605-ce7f-4c55-9695-ffd74a78ddcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:11.109019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1264/2536] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:11.117169 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66eadc-8cbc-404f-862f-0f57a55c621e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:11.117453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66eadc-8cbc-404f-862f-0f57a55c621e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:11.117721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66eadc-8cbc-404f-862f-0f57a55c621e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:11.138926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3d66eadc-8cbc-404f-862f-0f57a55c621e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:11.140162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3d66eadc-8cbc-404f-862f-0f57a55c621e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:11.149302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1273/2537] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:11.153932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1b605-ce7f-4c55-9695-ffd74a78ddcc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:11.164345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1b605-ce7f-4c55-9695-ffd74a78ddcc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:11.169893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e82e25-7527-4f24-a931-61efc8ec78fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:11.170507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e82e25-7527-4f24-a931-61efc8ec78fa admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:11.170779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e82e25-7527-4f24-a931-61efc8ec78fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:11.171039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e82e25-7527-4f24-a931-61efc8ec78fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:11.171681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e82e25-7527-4f24-a931-61efc8ec78fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:25:11.171944 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e82e25-7527-4f24-a931-61efc8ec78fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:11.173579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e82e25-7527-4f24-a931-61efc8ec78fa admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:11.186180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1265/2538] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:25:11.190804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66eadc-8cbc-404f-862f-0f57a55c621e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:11.202834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66eadc-8cbc-404f-862f-0f57a55c621e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:11.207847 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe0b8-e331-4eed-8fc4-ae4123119324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:11.208460 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe0b8-e331-4eed-8fc4-ae4123119324 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:11.208743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe0b8-e331-4eed-8fc4-ae4123119324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:11.209187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe0b8-e331-4eed-8fc4-ae4123119324 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:11.209830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32afe0b8-e331-4eed-8fc4-ae4123119324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:11.212046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32afe0b8-e331-4eed-8fc4-ae4123119324 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:11.244983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1274/2539] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:11 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:25:11.249496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e82e25-7527-4f24-a931-61efc8ec78fa admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:11.260980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e82e25-7527-4f24-a931-61efc8ec78fa admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:11.293236 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:11.335393 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf32a8-5f93-4355-ad10-221306ba94de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:11.335993 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf32a8-5f93-4355-ad10-221306ba94de admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:11.336380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf32a8-5f93-4355-ad10-221306ba94de admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:11.336696 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf32a8-5f93-4355-ad10-221306ba94de admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:11.337344 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf32a8-5f93-4355-ad10-221306ba94de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:11.339272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf32a8-5f93-4355-ad10-221306ba94de admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:11.364406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1266/2540] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:11 2022] POST /identity/v3/users => generated 284 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:11.369506 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe0b8-e331-4eed-8fc4-ae4123119324 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:11.380241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe0b8-e331-4eed-8fc4-ae4123119324 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:11.385981 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50650782-3d6f-43a5-8ba5-bb9927af149c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:11.386777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50650782-3d6f-43a5-8ba5-bb9927af149c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:11.387205 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50650782-3d6f-43a5-8ba5-bb9927af149c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:11.387590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50650782-3d6f-43a5-8ba5-bb9927af149c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:11.388540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50650782-3d6f-43a5-8ba5-bb9927af149c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:11.390923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50650782-3d6f-43a5-8ba5-bb9927af149c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:11.412469 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1275/2541] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:11.416917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf32a8-5f93-4355-ad10-221306ba94de admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:11.431132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf32a8-5f93-4355-ad10-221306ba94de admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:11.436401 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ad313-14b5-4093-a52e-7cc300c76a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:11.437060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ad313-14b5-4093-a52e-7cc300c76a88 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:11.437336 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ad313-14b5-4093-a52e-7cc300c76a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:11.437601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ad313-14b5-4093-a52e-7cc300c76a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:11.438239 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ad313-14b5-4093-a52e-7cc300c76a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:11.439850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ad313-14b5-4093-a52e-7cc300c76a88 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:11.451299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1267/2542] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:11.456159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50650782-3d6f-43a5-8ba5-bb9927af149c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:11.479146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50650782-3d6f-43a5-8ba5-bb9927af149c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:11.484404 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c364b-b580-4e8a-a777-2f3fee2ea3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:11.485161 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c364b-b580-4e8a-a777-2f3fee2ea3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:11.485439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c364b-b580-4e8a-a777-2f3fee2ea3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:11.485710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c364b-b580-4e8a-a777-2f3fee2ea3d0 admin admin] PATH_INFO: `/v3/projects/ebc08877c12d41399c685e4b9e7ce7a4/users/506f6a4f519c4a6e9873ed6b8d91f64e/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:11.501677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c364b-b580-4e8a-a777-2f3fee2ea3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc08877c12d41399c685e4b9e7ce7a4, user_id=506f6a4f519c4a6e9873ed6b8d91f64e, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:11.504483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c364b-b580-4e8a-a777-2f3fee2ea3d0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:11.522788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1276/2543] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:11 2022] PUT /identity/v3/projects/ebc08877c12d41399c685e4b9e7ce7a4/users/506f6a4f519c4a6e9873ed6b8d91f64e/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:11.538588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e14c42-2435-477a-8e47-22418af5197c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:11.538854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e14c42-2435-477a-8e47-22418af5197c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:11.539115 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e14c42-2435-477a-8e47-22418af5197c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:11.573347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-08e14c42-2435-477a-8e47-22418af5197c None None] MFA Rules not processed for user `506f6a4f519c4a6e9873ed6b8d91f64e`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:11.574570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-08e14c42-2435-477a-8e47-22418af5197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:11.643959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1268/2544] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:11.648952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c364b-b580-4e8a-a777-2f3fee2ea3d0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:11.720874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c364b-b580-4e8a-a777-2f3fee2ea3d0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:11.728770 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85254eb-85cc-4ed2-ae1e-dfec25e9422d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:11.729594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85254eb-85cc-4ed2-ae1e-dfec25e9422d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:11.730028 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85254eb-85cc-4ed2-ae1e-dfec25e9422d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:11.730488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85254eb-85cc-4ed2-ae1e-dfec25e9422d admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:11.731580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85254eb-85cc-4ed2-ae1e-dfec25e9422d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:11.734484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85254eb-85cc-4ed2-ae1e-dfec25e9422d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:11.763903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1277/2545] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:11 2022] POST /identity/v3/projects => generated 369 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:11.768860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e14c42-2435-477a-8e47-22418af5197c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:11.780507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e14c42-2435-477a-8e47-22418af5197c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:11.843142 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5c01b-0c31-4cb1-89a7-0d53bfd667fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:11.843755 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5c01b-0c31-4cb1-89a7-0d53bfd667fa admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:11.844055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5c01b-0c31-4cb1-89a7-0d53bfd667fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:11.844329 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5c01b-0c31-4cb1-89a7-0d53bfd667fa admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:11.844997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5c01b-0c31-4cb1-89a7-0d53bfd667fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:11.847007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5c01b-0c31-4cb1-89a7-0d53bfd667fa admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:11.871005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1269/2546] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:11 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:25:11.875725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85254eb-85cc-4ed2-ae1e-dfec25e9422d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:11.889216 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85254eb-85cc-4ed2-ae1e-dfec25e9422d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:11.896482 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdf593-52df-45a5-8043-bc3d13a0ac32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:11.897267 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdf593-52df-45a5-8043-bc3d13a0ac32 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:11.897545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdf593-52df-45a5-8043-bc3d13a0ac32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:11.897813 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdf593-52df-45a5-8043-bc3d13a0ac32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:11.898491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fdf593-52df-45a5-8043-bc3d13a0ac32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:11.900152 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fdf593-52df-45a5-8043-bc3d13a0ac32 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:11.912193 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1278/2547] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:11.916916 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5c01b-0c31-4cb1-89a7-0d53bfd667fa admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:11.928864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5c01b-0c31-4cb1-89a7-0d53bfd667fa admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:11.934601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9da326-2ef1-45a5-bb98-9e7ee735393f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:11.935493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9da326-2ef1-45a5-bb98-9e7ee735393f admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:11.936395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9da326-2ef1-45a5-bb98-9e7ee735393f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:11.936395 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9da326-2ef1-45a5-bb98-9e7ee735393f admin admin] PATH_INFO: `/v3/projects/7b2cacea8c14479181c5a0bbef77e033/users/35b5148e50d1441b958f180e8388e0f2/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:11.953590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9da326-2ef1-45a5-bb98-9e7ee735393f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2cacea8c14479181c5a0bbef77e033, user_id=35b5148e50d1441b958f180e8388e0f2, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:11.956285 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9da326-2ef1-45a5-bb98-9e7ee735393f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:11.971843 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1270/2548] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:11 2022] PUT /identity/v3/projects/7b2cacea8c14479181c5a0bbef77e033/users/35b5148e50d1441b958f180e8388e0f2/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:11.975845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdf593-52df-45a5-8043-bc3d13a0ac32 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:11.989572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdf593-52df-45a5-8043-bc3d13a0ac32 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:12.010880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:12.051227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951f745-dcab-479b-b2ab-a562f28e23f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:12.051783 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951f745-dcab-479b-b2ab-a562f28e23f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:12.052072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951f745-dcab-479b-b2ab-a562f28e23f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:12.052321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951f745-dcab-479b-b2ab-a562f28e23f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:12.053001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951f745-dcab-479b-b2ab-a562f28e23f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:12.054628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951f745-dcab-479b-b2ab-a562f28e23f2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:12.065120 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1279/2549] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:12.069791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9da326-2ef1-45a5-bb98-9e7ee735393f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:12.075904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:12.083006 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9da326-2ef1-45a5-bb98-9e7ee735393f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:12.088191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099f619-01d1-4d0a-a7bb-4580b704c2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:12.088876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099f619-01d1-4d0a-a7bb-4580b704c2e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:12.089145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099f619-01d1-4d0a-a7bb-4580b704c2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:12.089413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099f619-01d1-4d0a-a7bb-4580b704c2e4 admin admin] PATH_INFO: `/v3/projects/7b2cacea8c14479181c5a0bbef77e033/users/35b5148e50d1441b958f180e8388e0f2/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:12.091305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099f619-01d1-4d0a-a7bb-4580b704c2e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2cacea8c14479181c5a0bbef77e033, user_id=35b5148e50d1441b958f180e8388e0f2, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:12.094020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099f619-01d1-4d0a-a7bb-4580b704c2e4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:12.105338 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1271/2550] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:12 2022] PUT /identity/v3/projects/7b2cacea8c14479181c5a0bbef77e033/users/35b5148e50d1441b958f180e8388e0f2/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:12.116083 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cddc28-0e50-4836-a42d-8dc6d3d08eee None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:12.116365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cddc28-0e50-4836-a42d-8dc6d3d08eee None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:12.116641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cddc28-0e50-4836-a42d-8dc6d3d08eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:12.152775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-96cddc28-0e50-4836-a42d-8dc6d3d08eee None None] MFA Rules not processed for user `35b5148e50d1441b958f180e8388e0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:12.154174 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-96cddc28-0e50-4836-a42d-8dc6d3d08eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:12.225880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1280/2551] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:12.237480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099f619-01d1-4d0a-a7bb-4580b704c2e4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:12.239274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3099f619-01d1-4d0a-a7bb-4580b704c2e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:12.301464 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099f619-01d1-4d0a-a7bb-4580b704c2e4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:12.305916 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b16e2c-137c-41bd-8f7e-764804d004cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:12.306532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b16e2c-137c-41bd-8f7e-764804d004cf service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:12.306809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b16e2c-137c-41bd-8f7e-764804d004cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:12.307075 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b16e2c-137c-41bd-8f7e-764804d004cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:12.316800 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b16e2c-137c-41bd-8f7e-764804d004cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:12.318240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b16e2c-137c-41bd-8f7e-764804d004cf service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:12.328739 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1272/2552] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:13.821367 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cddc28-0e50-4836-a42d-8dc6d3d08eee None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:13.890547 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cddc28-0e50-4836-a42d-8dc6d3d08eee None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:13.895704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:13.896351 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:13.896650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:13.897269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] PATH_INFO: `/v3/users/506f6a4f519c4a6e9873ed6b8d91f64e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:13.898336 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=506f6a4f519c4a6e9873ed6b8d91f64e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:13.900343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:13.953364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '506f6a4f519c4a6e9873ed6b8d91f64e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:13.954046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '506f6a4f519c4a6e9873ed6b8d91f64e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:13.955249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:13.963258 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '506f6a4f519c4a6e9873ed6b8d91f64e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:13.980748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '506f6a4f519c4a6e9873ed6b8d91f64e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:13.981472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '506f6a4f519c4a6e9873ed6b8d91f64e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:13.995204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1281/2553] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:13 2022] DELETE /identity/v3/users/506f6a4f519c4a6e9873ed6b8d91f64e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:14.000592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def9701-eefc-4c00-b673-a2678730d18e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:14.001081 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def9701-eefc-4c00-b673-a2678730d18e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:14.001177 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def9701-eefc-4c00-b673-a2678730d18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:14.022508 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5def9701-eefc-4c00-b673-a2678730d18e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:14.023741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5def9701-eefc-4c00-b673-a2678730d18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:14.087270 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1273/2554] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:14.097630 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:14.099287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:14.159488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc56749-4c79-47cd-8adc-6f56be4423b2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:14.163343 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ddf63-9ff3-4947-8e92-1920ebe7fb48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:14.163961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ddf63-9ff3-4947-8e92-1920ebe7fb48 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:14.164294 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ddf63-9ff3-4947-8e92-1920ebe7fb48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:14.164617 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ddf63-9ff3-4947-8e92-1920ebe7fb48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:14.174989 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ddf63-9ff3-4947-8e92-1920ebe7fb48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:14.176465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ddf63-9ff3-4947-8e92-1920ebe7fb48 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:14.186583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1282/2555] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:14.480792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def9701-eefc-4c00-b673-a2678730d18e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:14.482476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5def9701-eefc-4c00-b673-a2678730d18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:14.502778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def9701-eefc-4c00-b673-a2678730d18e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:14.507965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:14.508807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:14.509224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:14.509616 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] PATH_INFO: `/v3/projects/ebc08877c12d41399c685e4b9e7ce7a4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:14.510853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc08877c12d41399c685e4b9e7ce7a4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:14.512819 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:14.579974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc08877c12d41399c685e4b9e7ce7a4', 'request_id': 'req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:14.592141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc08877c12d41399c685e4b9e7ce7a4', 'request_id': 'req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:14.600412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc08877c12d41399c685e4b9e7ce7a4', 'request_id': 'req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:14.612007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] The token cache is being invalidate because project ebc08877c12d41399c685e4b9e7ce7a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:14.612287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:14.613047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:14.614635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1274/2556] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:14 2022] DELETE /identity/v3/projects/ebc08877c12d41399c685e4b9e7ce7a4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:14.619921 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ddf63-9ff3-4947-8e92-1920ebe7fb48 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:14.634100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ddf63-9ff3-4947-8e92-1920ebe7fb48 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:14.703876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8073d5-fca0-4f74-a82f-cb0edee3ea1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:14.704686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8073d5-fca0-4f74-a82f-cb0edee3ea1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:14.705039 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8073d5-fca0-4f74-a82f-cb0edee3ea1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:14.705352 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8073d5-fca0-4f74-a82f-cb0edee3ea1c admin admin] PATH_INFO: `/v3/users/35b5148e50d1441b958f180e8388e0f2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:14.706903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8073d5-fca0-4f74-a82f-cb0edee3ea1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=35b5148e50d1441b958f180e8388e0f2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:14.709649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8073d5-fca0-4f74-a82f-cb0edee3ea1c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:14.764687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ec8073d5-fca0-4f74-a82f-cb0edee3ea1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35b5148e50d1441b958f180e8388e0f2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:14.765432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ec8073d5-fca0-4f74-a82f-cb0edee3ea1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35b5148e50d1441b958f180e8388e0f2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:14.774771 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ec8073d5-fca0-4f74-a82f-cb0edee3ea1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35b5148e50d1441b958f180e8388e0f2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:14.786865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:14.789285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ec8073d5-fca0-4f74-a82f-cb0edee3ea1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35b5148e50d1441b958f180e8388e0f2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:14.790317 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ec8073d5-fca0-4f74-a82f-cb0edee3ea1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35b5148e50d1441b958f180e8388e0f2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:14.807633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1283/2557] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:14 2022] DELETE /identity/v3/users/35b5148e50d1441b958f180e8388e0f2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:15.109985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:15.111916 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:15.126902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb8d68-9d5a-460d-8bf4-ebdd33ce1058 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:15.142021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:15.200491 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:15.201309 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:15.201711 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:15.202067 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a admin admin] PATH_INFO: `/v3/projects/7b2cacea8c14479181c5a0bbef77e033` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:15.203358 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b2cacea8c14479181c5a0bbef77e033)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:15.205347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:15.269103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b2cacea8c14479181c5a0bbef77e033', 'request_id': 'req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:15.280184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b2cacea8c14479181c5a0bbef77e033', 'request_id': 'req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:15.290077 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b2cacea8c14479181c5a0bbef77e033', 'request_id': 'req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:15.302725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a admin admin] The token cache is being invalidate because project 7b2cacea8c14479181c5a0bbef77e033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:15.302990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:15.303756 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b263a1f0-8390-4c9c-9733-7da5ce53fe5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:15.305358 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1275/2558] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:15 2022] DELETE /identity/v3/projects/7b2cacea8c14479181c5a0bbef77e033 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:25:15.310588 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef0f3f-fd39-4308-bcb4-219f774e6cc9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:15.311108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef0f3f-fd39-4308-bcb4-219f774e6cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:15.311259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef0f3f-fd39-4308-bcb4-219f774e6cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:15.332711 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-68ef0f3f-fd39-4308-bcb4-219f774e6cc9 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:15.333949 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-68ef0f3f-fd39-4308-bcb4-219f774e6cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:15.405646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1284/2559] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:15.416138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf80431-cec8-46b9-8e96-b11ec9b200d9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:15.416420 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf80431-cec8-46b9-8e96-b11ec9b200d9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:15.416713 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf80431-cec8-46b9-8e96-b11ec9b200d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:15.438758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0cf80431-cec8-46b9-8e96-b11ec9b200d9 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:15.439911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0cf80431-cec8-46b9-8e96-b11ec9b200d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:15.449772 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1276/2560] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:15.456552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef0f3f-fd39-4308-bcb4-219f774e6cc9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:15.472303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef0f3f-fd39-4308-bcb4-219f774e6cc9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:15.479088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86a7b9-5024-439a-abc3-986f102186ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:15.479692 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86a7b9-5024-439a-abc3-986f102186ed admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:15.479970 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86a7b9-5024-439a-abc3-986f102186ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:15.480243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86a7b9-5024-439a-abc3-986f102186ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:15.480925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86a7b9-5024-439a-abc3-986f102186ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:25:15.481229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86a7b9-5024-439a-abc3-986f102186ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:15.482855 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86a7b9-5024-439a-abc3-986f102186ed admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:15.495566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1285/2561] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:25:15.499985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf80431-cec8-46b9-8e96-b11ec9b200d9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:15.509968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf80431-cec8-46b9-8e96-b11ec9b200d9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:15.515659 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305183c-f6f5-4ece-8afa-bcfa27605a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:15.516263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305183c-f6f5-4ece-8afa-bcfa27605a35 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:15.516569 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305183c-f6f5-4ece-8afa-bcfa27605a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:15.516890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305183c-f6f5-4ece-8afa-bcfa27605a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:15.517608 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305183c-f6f5-4ece-8afa-bcfa27605a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:15.519614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305183c-f6f5-4ece-8afa-bcfa27605a35 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:15.550107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1277/2562] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:15 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:15.554477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86a7b9-5024-439a-abc3-986f102186ed admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:15.564677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86a7b9-5024-439a-abc3-986f102186ed admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:15.627519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5dae0-3e41-4e0f-8978-d34c2aceed30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:15.628136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5dae0-3e41-4e0f-8978-d34c2aceed30 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:15.628414 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5dae0-3e41-4e0f-8978-d34c2aceed30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:15.628695 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5dae0-3e41-4e0f-8978-d34c2aceed30 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:15.629341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5dae0-3e41-4e0f-8978-d34c2aceed30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:15.631268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5dae0-3e41-4e0f-8978-d34c2aceed30 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:15.662353 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1286/2563] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:15 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:15.667442 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305183c-f6f5-4ece-8afa-bcfa27605a35 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:15.678522 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305183c-f6f5-4ece-8afa-bcfa27605a35 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:15.683834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d932ce-cf49-48de-a916-efd00a1b61f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:15.684458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d932ce-cf49-48de-a916-efd00a1b61f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:15.684769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d932ce-cf49-48de-a916-efd00a1b61f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:15.685042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d932ce-cf49-48de-a916-efd00a1b61f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:15.685686 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d932ce-cf49-48de-a916-efd00a1b61f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:15.687323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d932ce-cf49-48de-a916-efd00a1b61f2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:15.698532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1278/2564] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:15.703645 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5dae0-3e41-4e0f-8978-d34c2aceed30 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:15.714531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5dae0-3e41-4e0f-8978-d34c2aceed30 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:15.719753 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa038ce-c18f-42ac-b945-f06b88726928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:15.720362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa038ce-c18f-42ac-b945-f06b88726928 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:15.720646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa038ce-c18f-42ac-b945-f06b88726928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:15.720950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa038ce-c18f-42ac-b945-f06b88726928 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:15.721600 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa038ce-c18f-42ac-b945-f06b88726928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:15.723225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa038ce-c18f-42ac-b945-f06b88726928 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:15.734623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1287/2565] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:15.738896 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d932ce-cf49-48de-a916-efd00a1b61f2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:15.749550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d932ce-cf49-48de-a916-efd00a1b61f2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:15.754920 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30a9f0-a84e-4a53-9812-c6f5f8e2ce90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:15.755610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30a9f0-a84e-4a53-9812-c6f5f8e2ce90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:15.755879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30a9f0-a84e-4a53-9812-c6f5f8e2ce90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:15.756462 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30a9f0-a84e-4a53-9812-c6f5f8e2ce90 admin admin] PATH_INFO: `/v3/projects/1c2791bf61964bfaa6b47c6b076ff841/users/bf78336ea0374d5a9eaafe4759702633/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:15.771877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30a9f0-a84e-4a53-9812-c6f5f8e2ce90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2791bf61964bfaa6b47c6b076ff841, user_id=bf78336ea0374d5a9eaafe4759702633, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:15.774655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30a9f0-a84e-4a53-9812-c6f5f8e2ce90 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:15.791138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1279/2566] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:15 2022] PUT /identity/v3/projects/1c2791bf61964bfaa6b47c6b076ff841/users/bf78336ea0374d5a9eaafe4759702633/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:15.803061 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab507dec-faee-4880-a802-7440c0199d59 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:15.803344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab507dec-faee-4880-a802-7440c0199d59 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:15.803616 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab507dec-faee-4880-a802-7440c0199d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:15.832641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ab507dec-faee-4880-a802-7440c0199d59 None None] MFA Rules not processed for user `bf78336ea0374d5a9eaafe4759702633`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:15.833882 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ab507dec-faee-4880-a802-7440c0199d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:15.879278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:15.904833 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1288/2567] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:15.909640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30a9f0-a84e-4a53-9812-c6f5f8e2ce90 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:15.956730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:15.978524 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30a9f0-a84e-4a53-9812-c6f5f8e2ce90 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:15.984116 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d2458-d352-41fe-9dd7-2832dcaad921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:15.984758 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d2458-d352-41fe-9dd7-2832dcaad921 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:15.985043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d2458-d352-41fe-9dd7-2832dcaad921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:15.985314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d2458-d352-41fe-9dd7-2832dcaad921 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:15.985950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d2458-d352-41fe-9dd7-2832dcaad921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:15.987861 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d2458-d352-41fe-9dd7-2832dcaad921 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:16.017521 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1280/2568] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:15 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:16.021710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab507dec-faee-4880-a802-7440c0199d59 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:16.032999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab507dec-faee-4880-a802-7440c0199d59 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:16.095894 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188e3f9-6cc5-4b59-8c39-0c8077e87fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:16.096515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188e3f9-6cc5-4b59-8c39-0c8077e87fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:16.096825 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188e3f9-6cc5-4b59-8c39-0c8077e87fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:16.097096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188e3f9-6cc5-4b59-8c39-0c8077e87fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:16.097746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188e3f9-6cc5-4b59-8c39-0c8077e87fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:16.099721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188e3f9-6cc5-4b59-8c39-0c8077e87fc5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:16.124056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1289/2569] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:16 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:16.128827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d2458-d352-41fe-9dd7-2832dcaad921 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:16.140299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d2458-d352-41fe-9dd7-2832dcaad921 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:16.148534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef22e37-ad27-462a-ae2a-6c7445ed2438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:16.149270 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef22e37-ad27-462a-ae2a-6c7445ed2438 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:16.149538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef22e37-ad27-462a-ae2a-6c7445ed2438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:16.149822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef22e37-ad27-462a-ae2a-6c7445ed2438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:16.150450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef22e37-ad27-462a-ae2a-6c7445ed2438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:16.152090 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef22e37-ad27-462a-ae2a-6c7445ed2438 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:16.163798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1281/2570] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:16.168613 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188e3f9-6cc5-4b59-8c39-0c8077e87fc5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:16.179795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188e3f9-6cc5-4b59-8c39-0c8077e87fc5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:16.185812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017e34a-bd50-4cf2-a07e-b207dc6fe358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:16.186478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017e34a-bd50-4cf2-a07e-b207dc6fe358 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:16.186752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017e34a-bd50-4cf2-a07e-b207dc6fe358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:16.187016 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017e34a-bd50-4cf2-a07e-b207dc6fe358 admin admin] PATH_INFO: `/v3/projects/7f0489aecd8d469c967934bb87629407/users/a11397f5fb4e44bcb6766fc1b731de18/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:16.204340 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017e34a-bd50-4cf2-a07e-b207dc6fe358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0489aecd8d469c967934bb87629407, user_id=a11397f5fb4e44bcb6766fc1b731de18, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:16.207046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017e34a-bd50-4cf2-a07e-b207dc6fe358 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:16.223136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1290/2571] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:16 2022] PUT /identity/v3/projects/7f0489aecd8d469c967934bb87629407/users/a11397f5fb4e44bcb6766fc1b731de18/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:16.227688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef22e37-ad27-462a-ae2a-6c7445ed2438 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:16.238137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef22e37-ad27-462a-ae2a-6c7445ed2438 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:16.296927 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cccb6dc-404e-455e-a7cb-960a0f654809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:16.297333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccb6dc-404e-455e-a7cb-960a0f654809 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:16.297612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccb6dc-404e-455e-a7cb-960a0f654809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:16.297870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccb6dc-404e-455e-a7cb-960a0f654809 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:16.298503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cccb6dc-404e-455e-a7cb-960a0f654809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:16.300105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cccb6dc-404e-455e-a7cb-960a0f654809 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:16.312232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1282/2572] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:16.316752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017e34a-bd50-4cf2-a07e-b207dc6fe358 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:16.328129 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017e34a-bd50-4cf2-a07e-b207dc6fe358 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:16.333381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756d1a5-33ea-4e8a-ae2c-3604aae6edb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:16.334050 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756d1a5-33ea-4e8a-ae2c-3604aae6edb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:16.334325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756d1a5-33ea-4e8a-ae2c-3604aae6edb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:16.334593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756d1a5-33ea-4e8a-ae2c-3604aae6edb7 admin admin] PATH_INFO: `/v3/projects/7f0489aecd8d469c967934bb87629407/users/a11397f5fb4e44bcb6766fc1b731de18/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:16.336831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756d1a5-33ea-4e8a-ae2c-3604aae6edb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0489aecd8d469c967934bb87629407, user_id=a11397f5fb4e44bcb6766fc1b731de18, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:16.339514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756d1a5-33ea-4e8a-ae2c-3604aae6edb7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:16.351993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1291/2573] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:16 2022] PUT /identity/v3/projects/7f0489aecd8d469c967934bb87629407/users/a11397f5fb4e44bcb6766fc1b731de18/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:16.363480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71310a53-27ed-48b4-84fc-5867e3cd7f3e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:16.363749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71310a53-27ed-48b4-84fc-5867e3cd7f3e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:16.364034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71310a53-27ed-48b4-84fc-5867e3cd7f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:16.397364 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-71310a53-27ed-48b4-84fc-5867e3cd7f3e None None] MFA Rules not processed for user `a11397f5fb4e44bcb6766fc1b731de18`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:16.399096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-71310a53-27ed-48b4-84fc-5867e3cd7f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:16.471600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1283/2574] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:16.488136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756d1a5-33ea-4e8a-ae2c-3604aae6edb7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:16.491344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5756d1a5-33ea-4e8a-ae2c-3604aae6edb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:16.556009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756d1a5-33ea-4e8a-ae2c-3604aae6edb7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:16.559775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e64d41-d111-47ab-a203-0a10c78a1941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:16.560535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e64d41-d111-47ab-a203-0a10c78a1941 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:16.560843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e64d41-d111-47ab-a203-0a10c78a1941 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:16.561115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e64d41-d111-47ab-a203-0a10c78a1941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:16.570569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e64d41-d111-47ab-a203-0a10c78a1941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:16.572046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e64d41-d111-47ab-a203-0a10c78a1941 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:16.582181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1292/2575] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:16.911806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71310a53-27ed-48b4-84fc-5867e3cd7f3e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:16.922066 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71310a53-27ed-48b4-84fc-5867e3cd7f3e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:16.927986 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41457a-410f-4a55-9051-3a51741b87f9 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a11397f5fb4e44bcb6766fc1b731de18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f0489aecd8d469c967934bb87629407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:16.928620 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41457a-410f-4a55-9051-3a51741b87f9 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:16.928929 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41457a-410f-4a55-9051-3a51741b87f9 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:16.929200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41457a-410f-4a55-9051-3a51741b87f9 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:16.929860 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41457a-410f-4a55-9051-3a51741b87f9 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:25:16.930167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41457a-410f-4a55-9051-3a51741b87f9 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:16.931789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41457a-410f-4a55-9051-3a51741b87f9 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:16.947083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1284/2576] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:25:16.953601 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e64d41-d111-47ab-a203-0a10c78a1941 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:16.966696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e64d41-d111-47ab-a203-0a10c78a1941 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:16.973508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6d47f-88df-4500-9a48-c45ac18ca4b6 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a11397f5fb4e44bcb6766fc1b731de18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f0489aecd8d469c967934bb87629407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:16.974185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6d47f-88df-4500-9a48-c45ac18ca4b6 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:16.974497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6d47f-88df-4500-9a48-c45ac18ca4b6 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:16.974791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6d47f-88df-4500-9a48-c45ac18ca4b6 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:16.975486 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6d47f-88df-4500-9a48-c45ac18ca4b6 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:16.977576 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6d47f-88df-4500-9a48-c45ac18ca4b6 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:17.008796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1293/2577] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:16 2022] POST /identity/v3/projects => generated 353 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:17.013355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41457a-410f-4a55-9051-3a51741b87f9 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:17.024896 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41457a-410f-4a55-9051-3a51741b87f9 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:17.049854 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:17.092244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d79cf8-99b9-4c12-92b9-8d67c437c8da tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a11397f5fb4e44bcb6766fc1b731de18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f0489aecd8d469c967934bb87629407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:17.092784 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d79cf8-99b9-4c12-92b9-8d67c437c8da tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:17.093053 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d79cf8-99b9-4c12-92b9-8d67c437c8da tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:17.093310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d79cf8-99b9-4c12-92b9-8d67c437c8da tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:17.093963 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d79cf8-99b9-4c12-92b9-8d67c437c8da tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:25:17.094263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d79cf8-99b9-4c12-92b9-8d67c437c8da tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:17.095921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d79cf8-99b9-4c12-92b9-8d67c437c8da tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:17.109069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1285/2578] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:25:17.152481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6d47f-88df-4500-9a48-c45ac18ca4b6 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:17.166666 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6d47f-88df-4500-9a48-c45ac18ca4b6 tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:17.172796 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d997ec2-b448-4723-8e27-e07507ff6e5f tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a11397f5fb4e44bcb6766fc1b731de18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f0489aecd8d469c967934bb87629407', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:17.173444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d997ec2-b448-4723-8e27-e07507ff6e5f tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:17.173750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d997ec2-b448-4723-8e27-e07507ff6e5f tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:17.174002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d997ec2-b448-4723-8e27-e07507ff6e5f tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] PATH_INFO: `/v3/projects/e04a325cef284d43b26d28e6229222f2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:17.175230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d997ec2-b448-4723-8e27-e07507ff6e5f tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e04a325cef284d43b26d28e6229222f2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:17.177279 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d997ec2-b448-4723-8e27-e07507ff6e5f tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:17.190755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:17.232442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0d997ec2-b448-4723-8e27-e07507ff6e5f tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e04a325cef284d43b26d28e6229222f2', 'request_id': 'req-0d997ec2-b448-4723-8e27-e07507ff6e5f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:17.245264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0d997ec2-b448-4723-8e27-e07507ff6e5f tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e04a325cef284d43b26d28e6229222f2', 'request_id': 'req-0d997ec2-b448-4723-8e27-e07507ff6e5f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:17.254904 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0d997ec2-b448-4723-8e27-e07507ff6e5f tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e04a325cef284d43b26d28e6229222f2', 'request_id': 'req-0d997ec2-b448-4723-8e27-e07507ff6e5f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:17.268471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0d997ec2-b448-4723-8e27-e07507ff6e5f tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] The token cache is being invalidate because project e04a325cef284d43b26d28e6229222f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:17.268800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0d997ec2-b448-4723-8e27-e07507ff6e5f tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:17.269465 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0d997ec2-b448-4723-8e27-e07507ff6e5f tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:17.271017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1294/2579] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:17 2022] DELETE /identity/v3/projects/e04a325cef284d43b26d28e6229222f2 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:17.497847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d79cf8-99b9-4c12-92b9-8d67c437c8da tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:17.499887 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-34d79cf8-99b9-4c12-92b9-8d67c437c8da tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:17.571286 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d79cf8-99b9-4c12-92b9-8d67c437c8da tempest-VolumeQuotaClassesTest-764026400 tempest-VolumeQuotaClassesTest-764026400-project-admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:17.642419 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:17.643056 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:17.643333 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:17.643604 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] PATH_INFO: `/v3/users/bf78336ea0374d5a9eaafe4759702633` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:17.644928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf78336ea0374d5a9eaafe4759702633)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:17.646912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:17.713037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf78336ea0374d5a9eaafe4759702633'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:17.713671 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf78336ea0374d5a9eaafe4759702633'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:17.722353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf78336ea0374d5a9eaafe4759702633'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:17.732622 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:17.738572 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf78336ea0374d5a9eaafe4759702633'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:17.739502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf78336ea0374d5a9eaafe4759702633'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:17.756622 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1286/2580] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:17 2022] DELETE /identity/v3/users/bf78336ea0374d5a9eaafe4759702633 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:17.763418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd17643-0deb-475b-bf98-71e9b46002c7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:17.763906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd17643-0deb-475b-bf98-71e9b46002c7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:17.764001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd17643-0deb-475b-bf98-71e9b46002c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:17.790445 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-dfd17643-0deb-475b-bf98-71e9b46002c7 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:17.791664 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dfd17643-0deb-475b-bf98-71e9b46002c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:17.858424 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1295/2581] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:17.869030 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:17.871406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:17.933796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a1760-47c3-4c24-a455-0682acb96874 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:17.937800 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e70b64-56ca-4ac7-a3da-65a11a53d44a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:17.938402 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e70b64-56ca-4ac7-a3da-65a11a53d44a service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:17.938679 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e70b64-56ca-4ac7-a3da-65a11a53d44a service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:17.938948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e70b64-56ca-4ac7-a3da-65a11a53d44a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:17.949229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e70b64-56ca-4ac7-a3da-65a11a53d44a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:17.950665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e70b64-56ca-4ac7-a3da-65a11a53d44a service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:17.960811 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1287/2582] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:18.250265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd17643-0deb-475b-bf98-71e9b46002c7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:18.251705 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dfd17643-0deb-475b-bf98-71e9b46002c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:18.265058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd17643-0deb-475b-bf98-71e9b46002c7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:18.270060 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:18.270707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:18.271023 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:18.271278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] PATH_INFO: `/v3/projects/1c2791bf61964bfaa6b47c6b076ff841` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:18.272427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2791bf61964bfaa6b47c6b076ff841)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:18.274647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:18.327028 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2791bf61964bfaa6b47c6b076ff841', 'request_id': 'req-c4aa2057-e871-43b5-8236-1fc71696ebe1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:18.338348 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2791bf61964bfaa6b47c6b076ff841', 'request_id': 'req-c4aa2057-e871-43b5-8236-1fc71696ebe1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:18.344690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2791bf61964bfaa6b47c6b076ff841', 'request_id': 'req-c4aa2057-e871-43b5-8236-1fc71696ebe1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:18.358036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] The token cache is being invalidate because project 1c2791bf61964bfaa6b47c6b076ff841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:18.358539 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:18.359377 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:18.361021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1296/2583] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:18 2022] DELETE /identity/v3/projects/1c2791bf61964bfaa6b47c6b076ff841 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:18.365954 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e70b64-56ca-4ac7-a3da-65a11a53d44a service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:18.376549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e70b64-56ca-4ac7-a3da-65a11a53d44a service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:18.443293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab85d40-3263-4427-8ba3-3684900da30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:18.443933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab85d40-3263-4427-8ba3-3684900da30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:18.444204 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab85d40-3263-4427-8ba3-3684900da30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:18.444500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab85d40-3263-4427-8ba3-3684900da30d admin admin] PATH_INFO: `/v3/users/a11397f5fb4e44bcb6766fc1b731de18` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:18.446034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab85d40-3263-4427-8ba3-3684900da30d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a11397f5fb4e44bcb6766fc1b731de18)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:18.448061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab85d40-3263-4427-8ba3-3684900da30d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:18.498015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dab85d40-3263-4427-8ba3-3684900da30d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a11397f5fb4e44bcb6766fc1b731de18'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:18.499083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dab85d40-3263-4427-8ba3-3684900da30d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a11397f5fb4e44bcb6766fc1b731de18'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:18.505828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dab85d40-3263-4427-8ba3-3684900da30d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a11397f5fb4e44bcb6766fc1b731de18'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:18.520230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dab85d40-3263-4427-8ba3-3684900da30d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a11397f5fb4e44bcb6766fc1b731de18'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:18.520933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dab85d40-3263-4427-8ba3-3684900da30d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a11397f5fb4e44bcb6766fc1b731de18'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:18.534039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1288/2584] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:18 2022] DELETE /identity/v3/users/a11397f5fb4e44bcb6766fc1b731de18 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:18.839055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:18.840805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:18.858888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2057-e871-43b5-8236-1fc71696ebe1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:18.881360 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:18.925572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d22527-0b1e-49f6-a712-2ec52bca1b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:18.926215 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d22527-0b1e-49f6-a712-2ec52bca1b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:18.926492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d22527-0b1e-49f6-a712-2ec52bca1b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:18.926761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d22527-0b1e-49f6-a712-2ec52bca1b0d admin admin] PATH_INFO: `/v3/projects/7f0489aecd8d469c967934bb87629407` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:18.928011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d22527-0b1e-49f6-a712-2ec52bca1b0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f0489aecd8d469c967934bb87629407)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:18.929877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d22527-0b1e-49f6-a712-2ec52bca1b0d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:18.982779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-27d22527-0b1e-49f6-a712-2ec52bca1b0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f0489aecd8d469c967934bb87629407', 'request_id': 'req-27d22527-0b1e-49f6-a712-2ec52bca1b0d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:18.989443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-27d22527-0b1e-49f6-a712-2ec52bca1b0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f0489aecd8d469c967934bb87629407', 'request_id': 'req-27d22527-0b1e-49f6-a712-2ec52bca1b0d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:18.998746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-27d22527-0b1e-49f6-a712-2ec52bca1b0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f0489aecd8d469c967934bb87629407', 'request_id': 'req-27d22527-0b1e-49f6-a712-2ec52bca1b0d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:19.013982 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-27d22527-0b1e-49f6-a712-2ec52bca1b0d admin admin] The token cache is being invalidate because project 7f0489aecd8d469c967934bb87629407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:19.014182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-27d22527-0b1e-49f6-a712-2ec52bca1b0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:19.014834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-27d22527-0b1e-49f6-a712-2ec52bca1b0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:19.016345 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1297/2585] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:18 2022] DELETE /identity/v3/projects/7f0489aecd8d469c967934bb87629407 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:19.021818 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e117d5-1e31-4a35-bb56-ebb455a415dc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:19.022367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e117d5-1e31-4a35-bb56-ebb455a415dc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:19.022367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e117d5-1e31-4a35-bb56-ebb455a415dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:19.037302 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:19.044622 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f4e117d5-1e31-4a35-bb56-ebb455a415dc None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:19.046610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f4e117d5-1e31-4a35-bb56-ebb455a415dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:19.108241 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1289/2586] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:19.116684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b4562-a38c-4100-ae01-c667be3fbf1c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:19.116990 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b4562-a38c-4100-ae01-c667be3fbf1c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:19.117244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b4562-a38c-4100-ae01-c667be3fbf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:19.138194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-634b4562-a38c-4100-ae01-c667be3fbf1c None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:19.139412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-634b4562-a38c-4100-ae01-c667be3fbf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:19.148565 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1298/2587] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:19.153564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e117d5-1e31-4a35-bb56-ebb455a415dc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:19.164824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e117d5-1e31-4a35-bb56-ebb455a415dc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:19.169735 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ed732-d096-4327-ad74-a4230cc73fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:19.170326 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ed732-d096-4327-ad74-a4230cc73fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:19.170603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ed732-d096-4327-ad74-a4230cc73fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:19.170866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ed732-d096-4327-ad74-a4230cc73fa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:19.171503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ed732-d096-4327-ad74-a4230cc73fa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:25:19.171796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ed732-d096-4327-ad74-a4230cc73fa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:19.173420 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ed732-d096-4327-ad74-a4230cc73fa9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:19.185382 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1290/2588] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:25:19.189950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b4562-a38c-4100-ae01-c667be3fbf1c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:19.200186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b4562-a38c-4100-ae01-c667be3fbf1c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:19.206619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b411617-0c04-4131-9e91-884aba0b5bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:19.207232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b411617-0c04-4131-9e91-884aba0b5bbe admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:19.207514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b411617-0c04-4131-9e91-884aba0b5bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:19.207784 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b411617-0c04-4131-9e91-884aba0b5bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:19.208419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b411617-0c04-4131-9e91-884aba0b5bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:19.210448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b411617-0c04-4131-9e91-884aba0b5bbe admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:19.241826 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1299/2589] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:19 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:19.248421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ed732-d096-4327-ad74-a4230cc73fa9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:19.263536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ed732-d096-4327-ad74-a4230cc73fa9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:19.331795 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023ad67-8813-44de-8a21-d0355d479fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:19.332350 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023ad67-8813-44de-8a21-d0355d479fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:19.332629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023ad67-8813-44de-8a21-d0355d479fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:19.332926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023ad67-8813-44de-8a21-d0355d479fe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:19.333568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023ad67-8813-44de-8a21-d0355d479fe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:19.335532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023ad67-8813-44de-8a21-d0355d479fe5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:19.360844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1291/2590] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:19 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:19.365172 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b411617-0c04-4131-9e91-884aba0b5bbe admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:19.377779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b411617-0c04-4131-9e91-884aba0b5bbe admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:19.383936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c561eac-1811-42dd-bee6-d9d367b7b675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:19.384559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c561eac-1811-42dd-bee6-d9d367b7b675 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:19.384857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c561eac-1811-42dd-bee6-d9d367b7b675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:19.385130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c561eac-1811-42dd-bee6-d9d367b7b675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:19.385841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c561eac-1811-42dd-bee6-d9d367b7b675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:19.387753 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c561eac-1811-42dd-bee6-d9d367b7b675 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:19.398647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1300/2591] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:19.404299 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023ad67-8813-44de-8a21-d0355d479fe5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:19.415453 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023ad67-8813-44de-8a21-d0355d479fe5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:19.421021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d93063-1d99-4f95-a5ec-74561c53f08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:19.421619 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d93063-1d99-4f95-a5ec-74561c53f08a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:19.421895 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d93063-1d99-4f95-a5ec-74561c53f08a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:19.422158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d93063-1d99-4f95-a5ec-74561c53f08a admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:19.422801 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d93063-1d99-4f95-a5ec-74561c53f08a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:19.424398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d93063-1d99-4f95-a5ec-74561c53f08a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:19.435628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1292/2592] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:19.440689 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c561eac-1811-42dd-bee6-d9d367b7b675 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:19.455400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c561eac-1811-42dd-bee6-d9d367b7b675 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:19.464013 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992df09-2e80-44a7-8cca-acdfbb13e018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:19.464696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992df09-2e80-44a7-8cca-acdfbb13e018 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:19.464983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992df09-2e80-44a7-8cca-acdfbb13e018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:19.465259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992df09-2e80-44a7-8cca-acdfbb13e018 admin admin] PATH_INFO: `/v3/projects/274f07625f554c829d1ee807d2f63ece/users/dc4a6cc47aa94cb6add6a712ae922481/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:19.482194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992df09-2e80-44a7-8cca-acdfbb13e018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=274f07625f554c829d1ee807d2f63ece, user_id=dc4a6cc47aa94cb6add6a712ae922481, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:19.484924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992df09-2e80-44a7-8cca-acdfbb13e018 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:19.499376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1301/2593] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:19 2022] PUT /identity/v3/projects/274f07625f554c829d1ee807d2f63ece/users/dc4a6cc47aa94cb6add6a712ae922481/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:19.511016 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e3b44-d97f-44f3-a58f-e0e901978688 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:19.511560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e3b44-d97f-44f3-a58f-e0e901978688 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:19.511668 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e3b44-d97f-44f3-a58f-e0e901978688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:19.541192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d63e3b44-d97f-44f3-a58f-e0e901978688 None None] MFA Rules not processed for user `dc4a6cc47aa94cb6add6a712ae922481`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:19.542562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d63e3b44-d97f-44f3-a58f-e0e901978688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:19.615202 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1293/2594] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:19.622002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992df09-2e80-44a7-8cca-acdfbb13e018 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:19.678519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:19.695688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992df09-2e80-44a7-8cca-acdfbb13e018 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:19.700426 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ccf2d-5648-4130-8c8a-08ce84f4218a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:19.701073 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ccf2d-5648-4130-8c8a-08ce84f4218a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:19.701354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ccf2d-5648-4130-8c8a-08ce84f4218a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:19.701628 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ccf2d-5648-4130-8c8a-08ce84f4218a admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:19.702272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ccf2d-5648-4130-8c8a-08ce84f4218a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:19.704235 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ccf2d-5648-4130-8c8a-08ce84f4218a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:19.733653 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1302/2595] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:19 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:19.737987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e3b44-d97f-44f3-a58f-e0e901978688 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:19.744747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:19.752198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e3b44-d97f-44f3-a58f-e0e901978688 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:19.810853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbe0cb-aa6e-49da-814c-380a515e3778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:19.811460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbe0cb-aa6e-49da-814c-380a515e3778 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:19.811735 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbe0cb-aa6e-49da-814c-380a515e3778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:19.812001 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbe0cb-aa6e-49da-814c-380a515e3778 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:19.812619 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbe0cb-aa6e-49da-814c-380a515e3778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:19.814576 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbe0cb-aa6e-49da-814c-380a515e3778 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:19.836524 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1294/2596] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:19 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:19.841217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ccf2d-5648-4130-8c8a-08ce84f4218a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:19.852080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ccf2d-5648-4130-8c8a-08ce84f4218a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:19.857044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfcaee-57fc-4ee7-9c88-2c8dceb11543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:19.857653 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfcaee-57fc-4ee7-9c88-2c8dceb11543 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:19.857945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfcaee-57fc-4ee7-9c88-2c8dceb11543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:19.858201 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfcaee-57fc-4ee7-9c88-2c8dceb11543 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:19.858842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfcaee-57fc-4ee7-9c88-2c8dceb11543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:19.860449 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfcaee-57fc-4ee7-9c88-2c8dceb11543 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:19.871251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1303/2597] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:19.875771 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbe0cb-aa6e-49da-814c-380a515e3778 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:19.886989 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbe0cb-aa6e-49da-814c-380a515e3778 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:19.892685 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b42f2e-25a7-49de-aa2e-87986e71e3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:19.893343 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b42f2e-25a7-49de-aa2e-87986e71e3d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:19.893623 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b42f2e-25a7-49de-aa2e-87986e71e3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:19.893895 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b42f2e-25a7-49de-aa2e-87986e71e3d9 admin admin] PATH_INFO: `/v3/projects/dad32c7b514c4a41b58164409b4f4f76/users/302e8d4c78a3472db4d0b0af84b375dd/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:19.911665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b42f2e-25a7-49de-aa2e-87986e71e3d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad32c7b514c4a41b58164409b4f4f76, user_id=302e8d4c78a3472db4d0b0af84b375dd, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:19.914731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b42f2e-25a7-49de-aa2e-87986e71e3d9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:19.931702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1295/2598] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:19 2022] PUT /identity/v3/projects/dad32c7b514c4a41b58164409b4f4f76/users/302e8d4c78a3472db4d0b0af84b375dd/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:19.937830 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfcaee-57fc-4ee7-9c88-2c8dceb11543 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:19.950189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfcaee-57fc-4ee7-9c88-2c8dceb11543 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:20.008993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f817c2-959c-4677-9113-8f13888e0fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:20.009599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f817c2-959c-4677-9113-8f13888e0fee admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:20.009902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f817c2-959c-4677-9113-8f13888e0fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:20.010177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f817c2-959c-4677-9113-8f13888e0fee admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:20.010892 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f817c2-959c-4677-9113-8f13888e0fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:20.012522 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f817c2-959c-4677-9113-8f13888e0fee admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:20.022672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1304/2599] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:20.027131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b42f2e-25a7-49de-aa2e-87986e71e3d9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:20.038038 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b42f2e-25a7-49de-aa2e-87986e71e3d9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:20.044141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf6240-4630-4c4e-8183-2ba89082126f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:20.044838 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf6240-4630-4c4e-8183-2ba89082126f admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:20.045112 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf6240-4630-4c4e-8183-2ba89082126f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:20.045383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf6240-4630-4c4e-8183-2ba89082126f admin admin] PATH_INFO: `/v3/projects/dad32c7b514c4a41b58164409b4f4f76/users/302e8d4c78a3472db4d0b0af84b375dd/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:20.047380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf6240-4630-4c4e-8183-2ba89082126f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad32c7b514c4a41b58164409b4f4f76, user_id=302e8d4c78a3472db4d0b0af84b375dd, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:20.050019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf6240-4630-4c4e-8183-2ba89082126f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:20.060891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1296/2600] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:20 2022] PUT /identity/v3/projects/dad32c7b514c4a41b58164409b4f4f76/users/302e8d4c78a3472db4d0b0af84b375dd/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:20.071575 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948e731-bca5-4e5e-9bd6-062b43e3baf4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:20.071860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948e731-bca5-4e5e-9bd6-062b43e3baf4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:20.072151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948e731-bca5-4e5e-9bd6-062b43e3baf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:20.102233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f948e731-bca5-4e5e-9bd6-062b43e3baf4 None None] MFA Rules not processed for user `302e8d4c78a3472db4d0b0af84b375dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:20.103352 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f948e731-bca5-4e5e-9bd6-062b43e3baf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:20.174017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1305/2601] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:20.188001 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf6240-4630-4c4e-8183-2ba89082126f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:20.189957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2edf6240-4630-4c4e-8183-2ba89082126f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:20.253589 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf6240-4630-4c4e-8183-2ba89082126f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:20.257430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a49699-b110-4d1f-a652-7b5a79a10c17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:20.258025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a49699-b110-4d1f-a652-7b5a79a10c17 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:20.258289 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a49699-b110-4d1f-a652-7b5a79a10c17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:20.258551 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a49699-b110-4d1f-a652-7b5a79a10c17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:20.268090 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a49699-b110-4d1f-a652-7b5a79a10c17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:20.269560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a49699-b110-4d1f-a652-7b5a79a10c17 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:20.279115 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1297/2602] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:20.396390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948e731-bca5-4e5e-9bd6-062b43e3baf4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:20.406613 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948e731-bca5-4e5e-9bd6-062b43e3baf4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:20.410410 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bece412-ea39-49c4-a888-b0cbb6fbe186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:20.411010 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bece412-ea39-49c4-a888-b0cbb6fbe186 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:20.411285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bece412-ea39-49c4-a888-b0cbb6fbe186 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:20.411552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bece412-ea39-49c4-a888-b0cbb6fbe186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:20.454252 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:20.470660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bece412-ea39-49c4-a888-b0cbb6fbe186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:20.472151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bece412-ea39-49c4-a888-b0cbb6fbe186 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:20.481394 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1306/2603] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:23.116123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a49699-b110-4d1f-a652-7b5a79a10c17 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:23.176649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:23.181894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a49699-b110-4d1f-a652-7b5a79a10c17 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:23.187538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:23.188162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:23.188431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:23.188690 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] PATH_INFO: `/v3/users/dc4a6cc47aa94cb6add6a712ae922481` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:23.189866 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc4a6cc47aa94cb6add6a712ae922481)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:23.191749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:23.238559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc4a6cc47aa94cb6add6a712ae922481'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:23.239268 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc4a6cc47aa94cb6add6a712ae922481'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:23.246553 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc4a6cc47aa94cb6add6a712ae922481'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:23.258457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc4a6cc47aa94cb6add6a712ae922481'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:23.259089 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc4a6cc47aa94cb6add6a712ae922481'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:23.272712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1298/2604] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:23 2022] DELETE /identity/v3/users/dc4a6cc47aa94cb6add6a712ae922481 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:23.277584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcb8df-8c99-4754-a9eb-4e950617d915 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:23.278098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcb8df-8c99-4754-a9eb-4e950617d915 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:23.278198 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcb8df-8c99-4754-a9eb-4e950617d915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:23.298543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a3fcb8df-8c99-4754-a9eb-4e950617d915 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:23.299782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a3fcb8df-8c99-4754-a9eb-4e950617d915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:23.365263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1307/2605] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:23.376307 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:23.377980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:23.444932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7facb-b04c-478a-801b-8e15feaa3e16 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:23.449169 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e449b6c-ce01-4b14-8de3-777018deb1ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:23.449770 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e449b6c-ce01-4b14-8de3-777018deb1ff service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:23.450048 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e449b6c-ce01-4b14-8de3-777018deb1ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:23.450317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e449b6c-ce01-4b14-8de3-777018deb1ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:23.461126 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e449b6c-ce01-4b14-8de3-777018deb1ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:23.462575 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e449b6c-ce01-4b14-8de3-777018deb1ff service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:23.472532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1299/2606] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:23.753634 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcb8df-8c99-4754-a9eb-4e950617d915 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:23.755273 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a3fcb8df-8c99-4754-a9eb-4e950617d915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:23.769681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcb8df-8c99-4754-a9eb-4e950617d915 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:23.774794 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:23.775446 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:23.775721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:23.776064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] PATH_INFO: `/v3/projects/274f07625f554c829d1ee807d2f63ece` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:23.777230 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=274f07625f554c829d1ee807d2f63ece)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:23.779251 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:23.828775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '274f07625f554c829d1ee807d2f63ece', 'request_id': 'req-63b6223e-0575-4c40-b02b-be711a5999ba'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:23.834905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '274f07625f554c829d1ee807d2f63ece', 'request_id': 'req-63b6223e-0575-4c40-b02b-be711a5999ba'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:23.840489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '274f07625f554c829d1ee807d2f63ece', 'request_id': 'req-63b6223e-0575-4c40-b02b-be711a5999ba'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:23.854533 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] The token cache is being invalidate because project 274f07625f554c829d1ee807d2f63ece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:23.854832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:23.855457 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:23.857047 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1308/2607] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:23 2022] DELETE /identity/v3/projects/274f07625f554c829d1ee807d2f63ece => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:23.861260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e449b6c-ce01-4b14-8de3-777018deb1ff service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:23.871567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e449b6c-ce01-4b14-8de3-777018deb1ff service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:23.932601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050e889-b5b8-4c57-9f53-0498d61fde5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:23.933272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050e889-b5b8-4c57-9f53-0498d61fde5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:23.933550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050e889-b5b8-4c57-9f53-0498d61fde5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:23.933818 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050e889-b5b8-4c57-9f53-0498d61fde5d admin admin] PATH_INFO: `/v3/users/302e8d4c78a3472db4d0b0af84b375dd` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:23.935073 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050e889-b5b8-4c57-9f53-0498d61fde5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=302e8d4c78a3472db4d0b0af84b375dd)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:23.937046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050e889-b5b8-4c57-9f53-0498d61fde5d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:23.959082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:23.988019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f050e889-b5b8-4c57-9f53-0498d61fde5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '302e8d4c78a3472db4d0b0af84b375dd'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:23.988708 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f050e889-b5b8-4c57-9f53-0498d61fde5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '302e8d4c78a3472db4d0b0af84b375dd'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:23.995692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f050e889-b5b8-4c57-9f53-0498d61fde5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '302e8d4c78a3472db4d0b0af84b375dd'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:24.007957 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f050e889-b5b8-4c57-9f53-0498d61fde5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '302e8d4c78a3472db4d0b0af84b375dd'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:24.008983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f050e889-b5b8-4c57-9f53-0498d61fde5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '302e8d4c78a3472db4d0b0af84b375dd'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:24.023196 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1300/2608] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:23 2022] DELETE /identity/v3/users/302e8d4c78a3472db4d0b0af84b375dd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:24.306564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:24.308182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:24.321855 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6223e-0575-4c40-b02b-be711a5999ba admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:24.381691 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:24.382259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:24.382566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:24.382842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d admin admin] PATH_INFO: `/v3/projects/dad32c7b514c4a41b58164409b4f4f76` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:24.383987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad32c7b514c4a41b58164409b4f4f76)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:24.386162 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:24.395099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:24.436968 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad32c7b514c4a41b58164409b4f4f76', 'request_id': 'req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:24.443143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad32c7b514c4a41b58164409b4f4f76', 'request_id': 'req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:24.448624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad32c7b514c4a41b58164409b4f4f76', 'request_id': 'req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:24.465364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d admin admin] The token cache is being invalidate because project dad32c7b514c4a41b58164409b4f4f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:24.465653 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:24.466285 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8cb49e13-b941-4fbb-a0a1-1c0aad69e16d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:24.467812 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1309/2609] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:24 2022] DELETE /identity/v3/projects/dad32c7b514c4a41b58164409b4f4f76 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:24.473778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf1a46-8e93-4639-ab47-b30e78018d87 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:24.474539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf1a46-8e93-4639-ab47-b30e78018d87 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:24.474707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf1a46-8e93-4639-ab47-b30e78018d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:24.496425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4eaf1a46-8e93-4639-ab47-b30e78018d87 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:24.497726 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4eaf1a46-8e93-4639-ab47-b30e78018d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:24.562829 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1301/2610] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:25:24.572220 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d4326-99dc-40fe-9480-c09bcd5a8eb0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:24.572635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d4326-99dc-40fe-9480-c09bcd5a8eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:24.573091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d4326-99dc-40fe-9480-c09bcd5a8eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:24.595925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ed7d4326-99dc-40fe-9480-c09bcd5a8eb0 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:24.597829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ed7d4326-99dc-40fe-9480-c09bcd5a8eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:24.609740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1310/2611] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:24.614497 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf1a46-8e93-4639-ab47-b30e78018d87 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:24.625042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf1a46-8e93-4639-ab47-b30e78018d87 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:24.631441 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632799fa-4c10-47d2-bea7-90336a1fe093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:24.632108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632799fa-4c10-47d2-bea7-90336a1fe093 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:24.632325 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632799fa-4c10-47d2-bea7-90336a1fe093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:24.632596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632799fa-4c10-47d2-bea7-90336a1fe093 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:24.633289 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632799fa-4c10-47d2-bea7-90336a1fe093 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:25:24.633592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632799fa-4c10-47d2-bea7-90336a1fe093 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:24.635183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632799fa-4c10-47d2-bea7-90336a1fe093 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:24.647594 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1302/2612] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:25:24.652597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d4326-99dc-40fe-9480-c09bcd5a8eb0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:24.665654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d4326-99dc-40fe-9480-c09bcd5a8eb0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:24.672214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b818d-64c7-4ca3-85c3-e58a2e2f1b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:24.672864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b818d-64c7-4ca3-85c3-e58a2e2f1b86 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:24.673142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b818d-64c7-4ca3-85c3-e58a2e2f1b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:24.673409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b818d-64c7-4ca3-85c3-e58a2e2f1b86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:24.674046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b818d-64c7-4ca3-85c3-e58a2e2f1b86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:24.676017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b818d-64c7-4ca3-85c3-e58a2e2f1b86 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:24.711840 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1311/2613] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:24 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:24.716525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632799fa-4c10-47d2-bea7-90336a1fe093 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:24.726777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632799fa-4c10-47d2-bea7-90336a1fe093 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:24.785315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b6e49-5b10-437a-98e5-2518b39674bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:24.785919 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b6e49-5b10-437a-98e5-2518b39674bb admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:24.786302 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b6e49-5b10-437a-98e5-2518b39674bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:24.786592 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b6e49-5b10-437a-98e5-2518b39674bb admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:24.787225 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b6e49-5b10-437a-98e5-2518b39674bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:24.789190 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b6e49-5b10-437a-98e5-2518b39674bb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:24.812564 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1303/2614] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:24 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:24.816935 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b818d-64c7-4ca3-85c3-e58a2e2f1b86 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:24.827760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b818d-64c7-4ca3-85c3-e58a2e2f1b86 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:24.833751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc10b9-3fcf-4a43-ba7c-77e9c0e96553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:24.834375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc10b9-3fcf-4a43-ba7c-77e9c0e96553 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:24.834650 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc10b9-3fcf-4a43-ba7c-77e9c0e96553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:24.834925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc10b9-3fcf-4a43-ba7c-77e9c0e96553 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:24.835601 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc10b9-3fcf-4a43-ba7c-77e9c0e96553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:24.837295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc10b9-3fcf-4a43-ba7c-77e9c0e96553 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:24.847284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1312/2615] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:24.851662 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b6e49-5b10-437a-98e5-2518b39674bb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:24.862094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b6e49-5b10-437a-98e5-2518b39674bb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:24.867029 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251d9d9-db97-4bf4-96b0-ad3cdb6c2f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:24.867626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251d9d9-db97-4bf4-96b0-ad3cdb6c2f3b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:24.867906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251d9d9-db97-4bf4-96b0-ad3cdb6c2f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:24.868176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251d9d9-db97-4bf4-96b0-ad3cdb6c2f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:24.868850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251d9d9-db97-4bf4-96b0-ad3cdb6c2f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:24.870502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251d9d9-db97-4bf4-96b0-ad3cdb6c2f3b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:24.881472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1304/2616] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:24.886213 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc10b9-3fcf-4a43-ba7c-77e9c0e96553 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:24.896272 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc10b9-3fcf-4a43-ba7c-77e9c0e96553 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:24.901214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72592a-131a-4bc7-99f8-9cf9245675df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:24.901861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72592a-131a-4bc7-99f8-9cf9245675df admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:24.902132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72592a-131a-4bc7-99f8-9cf9245675df admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:24.902397 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72592a-131a-4bc7-99f8-9cf9245675df admin admin] PATH_INFO: `/v3/projects/828b884c419740c68c00837d8c73e3a1/users/fb22e9519ef249b8be695dd4391eaf8e/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:24.917022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72592a-131a-4bc7-99f8-9cf9245675df admin admin] RBAC: Authorizing `identity:create_grant(project_id=828b884c419740c68c00837d8c73e3a1, user_id=fb22e9519ef249b8be695dd4391eaf8e, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:24.920035 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72592a-131a-4bc7-99f8-9cf9245675df admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:24.935929 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1313/2617] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:24 2022] PUT /identity/v3/projects/828b884c419740c68c00837d8c73e3a1/users/fb22e9519ef249b8be695dd4391eaf8e/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:24.946311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92c627-0a48-4d20-b303-d812be991388 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:24.946722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92c627-0a48-4d20-b303-d812be991388 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:24.946908 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92c627-0a48-4d20-b303-d812be991388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:24.955365 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:24.979264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fc92c627-0a48-4d20-b303-d812be991388 None None] MFA Rules not processed for user `fb22e9519ef249b8be695dd4391eaf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:24.980533 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fc92c627-0a48-4d20-b303-d812be991388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:25.048022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1305/2618] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:25.062495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72592a-131a-4bc7-99f8-9cf9245675df admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:25.064682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0d72592a-131a-4bc7-99f8-9cf9245675df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:25.139024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72592a-131a-4bc7-99f8-9cf9245675df admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:25.142999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b1438-8474-49f4-bae6-45d381181f67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:25.143610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b1438-8474-49f4-bae6-45d381181f67 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:25.143894 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b1438-8474-49f4-bae6-45d381181f67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:25.144164 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b1438-8474-49f4-bae6-45d381181f67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:25.152878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b1438-8474-49f4-bae6-45d381181f67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:25.154341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b1438-8474-49f4-bae6-45d381181f67 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:25.163350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1314/2619] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:25.185837 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92c627-0a48-4d20-b303-d812be991388 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:25.253146 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92c627-0a48-4d20-b303-d812be991388 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:25.258414 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:25.259076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:25.259356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:25.259625 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] PATH_INFO: `/v3/users/fb22e9519ef249b8be695dd4391eaf8e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:25.261021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb22e9519ef249b8be695dd4391eaf8e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:25.263004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:25.316904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb22e9519ef249b8be695dd4391eaf8e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:25.317555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb22e9519ef249b8be695dd4391eaf8e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:25.324897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb22e9519ef249b8be695dd4391eaf8e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:25.337437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb22e9519ef249b8be695dd4391eaf8e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:25.338209 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb22e9519ef249b8be695dd4391eaf8e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:25.355500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1306/2620] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:25 2022] DELETE /identity/v3/users/fb22e9519ef249b8be695dd4391eaf8e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:25.360195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fbb52-60e5-4f3c-8163-4ed36a7d4df1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:25.360841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fbb52-60e5-4f3c-8163-4ed36a7d4df1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:25.360841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fbb52-60e5-4f3c-8163-4ed36a7d4df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:25.380635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7e7fbb52-60e5-4f3c-8163-4ed36a7d4df1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:25.381883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7e7fbb52-60e5-4f3c-8163-4ed36a7d4df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:25.413033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:25.443681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1315/2621] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:25.454089 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:25.455767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:25.471204 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:25.520295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749bd8c-3cdd-43c1-ba7e-cded6051b9ab admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:25.524515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760190f8-3d3c-4bce-b8be-6cf8122f8bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:25.525159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760190f8-3d3c-4bce-b8be-6cf8122f8bce service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:25.525437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760190f8-3d3c-4bce-b8be-6cf8122f8bce service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:25.525764 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760190f8-3d3c-4bce-b8be-6cf8122f8bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:25.535574 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760190f8-3d3c-4bce-b8be-6cf8122f8bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:25.537044 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760190f8-3d3c-4bce-b8be-6cf8122f8bce service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:25.552198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1307/2622] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:25.858983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fbb52-60e5-4f3c-8163-4ed36a7d4df1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:25.860539 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7e7fbb52-60e5-4f3c-8163-4ed36a7d4df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:25.873178 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fbb52-60e5-4f3c-8163-4ed36a7d4df1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:25.881707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:25.883074 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:25.883944 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:25.884556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0 admin admin] PATH_INFO: `/v3/projects/828b884c419740c68c00837d8c73e3a1` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:25.887439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=828b884c419740c68c00837d8c73e3a1)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:25.889417 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:25.939167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '828b884c419740c68c00837d8c73e3a1', 'request_id': 'req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:25.945224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '828b884c419740c68c00837d8c73e3a1', 'request_id': 'req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:25.950629 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '828b884c419740c68c00837d8c73e3a1', 'request_id': 'req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:25.964119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0 admin admin] The token cache is being invalidate because project 828b884c419740c68c00837d8c73e3a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:25.964402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:25.965029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2472c22d-ffcf-4919-8b0b-e794e8a51eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:25.966558 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1316/2623] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:25 2022] DELETE /identity/v3/projects/828b884c419740c68c00837d8c73e3a1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:25:25.972208 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744350d-dafe-454b-bb7c-849489145808 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:25.972809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744350d-dafe-454b-bb7c-849489145808 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:25.973190 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744350d-dafe-454b-bb7c-849489145808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:25.995156 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3744350d-dafe-454b-bb7c-849489145808 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:25.996702 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3744350d-dafe-454b-bb7c-849489145808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:26.068403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1308/2624] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:26.078033 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dedd1f-029e-4a69-b332-6a9a4575d922 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:26.078310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dedd1f-029e-4a69-b332-6a9a4575d922 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:26.078996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dedd1f-029e-4a69-b332-6a9a4575d922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:26.108101 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-52dedd1f-029e-4a69-b332-6a9a4575d922 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:26.109552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-52dedd1f-029e-4a69-b332-6a9a4575d922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:26.118659 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1317/2625] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:26.123808 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744350d-dafe-454b-bb7c-849489145808 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:26.138886 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744350d-dafe-454b-bb7c-849489145808 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:26.144685 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a0159-d6a9-4585-bfd7-4cb9fc16d6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:26.145465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a0159-d6a9-4585-bfd7-4cb9fc16d6bc admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:26.145812 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a0159-d6a9-4585-bfd7-4cb9fc16d6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:26.146189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a0159-d6a9-4585-bfd7-4cb9fc16d6bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:26.147008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a0159-d6a9-4585-bfd7-4cb9fc16d6bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:25:26.147308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a0159-d6a9-4585-bfd7-4cb9fc16d6bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:26.149218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a0159-d6a9-4585-bfd7-4cb9fc16d6bc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:26.162631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1309/2626] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:25:26.167522 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dedd1f-029e-4a69-b332-6a9a4575d922 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:26.177736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dedd1f-029e-4a69-b332-6a9a4575d922 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:26.182627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea58c0-8f09-4c86-bd79-918fe9305af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:26.183244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea58c0-8f09-4c86-bd79-918fe9305af0 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:26.183523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea58c0-8f09-4c86-bd79-918fe9305af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:26.183795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea58c0-8f09-4c86-bd79-918fe9305af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:26.184428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea58c0-8f09-4c86-bd79-918fe9305af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:26.186398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea58c0-8f09-4c86-bd79-918fe9305af0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:26.217860 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1318/2627] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:26 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:26.222376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a0159-d6a9-4585-bfd7-4cb9fc16d6bc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:26.232693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a0159-d6a9-4585-bfd7-4cb9fc16d6bc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:26.291086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4196111-1087-4bbb-b704-c9be39db4ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:26.291720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196111-1087-4bbb-b704-c9be39db4ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:26.291995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196111-1087-4bbb-b704-c9be39db4ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:26.292261 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196111-1087-4bbb-b704-c9be39db4ef5 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:26.292925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4196111-1087-4bbb-b704-c9be39db4ef5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:26.295198 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4196111-1087-4bbb-b704-c9be39db4ef5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:26.317962 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1310/2628] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:26 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:26.322470 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea58c0-8f09-4c86-bd79-918fe9305af0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:26.332925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea58c0-8f09-4c86-bd79-918fe9305af0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:26.337999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e0765-4da7-42ce-a358-7093df47a71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:26.338678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e0765-4da7-42ce-a358-7093df47a71a admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:26.338942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e0765-4da7-42ce-a358-7093df47a71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:26.339212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e0765-4da7-42ce-a358-7093df47a71a admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:26.339851 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e0765-4da7-42ce-a358-7093df47a71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:26.341498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e0765-4da7-42ce-a358-7093df47a71a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:26.351356 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1319/2629] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:26.355347 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4196111-1087-4bbb-b704-c9be39db4ef5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:26.366034 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4196111-1087-4bbb-b704-c9be39db4ef5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:26.371472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef8b5b-03b2-4e9c-a992-3c4842b4c591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:26.372070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef8b5b-03b2-4e9c-a992-3c4842b4c591 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:26.372345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef8b5b-03b2-4e9c-a992-3c4842b4c591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:26.372644 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef8b5b-03b2-4e9c-a992-3c4842b4c591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:26.373316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef8b5b-03b2-4e9c-a992-3c4842b4c591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:26.374927 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef8b5b-03b2-4e9c-a992-3c4842b4c591 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:26.389159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1311/2630] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:25:26.393478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e0765-4da7-42ce-a358-7093df47a71a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:26.406188 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e0765-4da7-42ce-a358-7093df47a71a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:26.411349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185d343-052e-4e15-a58f-7b2c87746ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:26.412023 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185d343-052e-4e15-a58f-7b2c87746ef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:26.412333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185d343-052e-4e15-a58f-7b2c87746ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:26.412612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185d343-052e-4e15-a58f-7b2c87746ef5 admin admin] PATH_INFO: `/v3/projects/eb07d4b53c7d44a3b47854f3f0d27b25/users/fd94232f6ecf4c8295ec13bd39502c54/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:26.429220 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185d343-052e-4e15-a58f-7b2c87746ef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb07d4b53c7d44a3b47854f3f0d27b25, user_id=fd94232f6ecf4c8295ec13bd39502c54, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:26.431894 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185d343-052e-4e15-a58f-7b2c87746ef5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:26.446670 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1320/2631] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:26 2022] PUT /identity/v3/projects/eb07d4b53c7d44a3b47854f3f0d27b25/users/fd94232f6ecf4c8295ec13bd39502c54/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:26.520224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c203272-3fad-497c-9038-c3676a5800b3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:26.520509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c203272-3fad-497c-9038-c3676a5800b3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:26.521107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c203272-3fad-497c-9038-c3676a5800b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:26.553306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7c203272-3fad-497c-9038-c3676a5800b3 None None] MFA Rules not processed for user `fd94232f6ecf4c8295ec13bd39502c54`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:26.555006 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7c203272-3fad-497c-9038-c3676a5800b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:26.565765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:26.629830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1312/2632] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:26.641680 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185d343-052e-4e15-a58f-7b2c87746ef5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:26.643742 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7185d343-052e-4e15-a58f-7b2c87746ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:26.709745 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185d343-052e-4e15-a58f-7b2c87746ef5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:26.713919 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c555a-3e4e-4c03-a5a3-0d10e2d88cef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:26.714534 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c555a-3e4e-4c03-a5a3-0d10e2d88cef service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:26.714809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c555a-3e4e-4c03-a5a3-0d10e2d88cef service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:26.715142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c555a-3e4e-4c03-a5a3-0d10e2d88cef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:26.728059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c555a-3e4e-4c03-a5a3-0d10e2d88cef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:26.729948 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c555a-3e4e-4c03-a5a3-0d10e2d88cef service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:26.733491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:26.741623 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1321/2633] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:36.278068 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c203272-3fad-497c-9038-c3676a5800b3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:36.343613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c203272-3fad-497c-9038-c3676a5800b3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:36.348211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:36.348883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:36.349146 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:36.349418 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] PATH_INFO: `/v3/users/fd94232f6ecf4c8295ec13bd39502c54` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:36.350551 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd94232f6ecf4c8295ec13bd39502c54)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:36.354587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:36.427596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd94232f6ecf4c8295ec13bd39502c54'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:36.428270 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd94232f6ecf4c8295ec13bd39502c54'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:36.438119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd94232f6ecf4c8295ec13bd39502c54'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:36.458851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd94232f6ecf4c8295ec13bd39502c54'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:36.460277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd94232f6ecf4c8295ec13bd39502c54'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:36.477525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1313/2634] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:36 2022] DELETE /identity/v3/users/fd94232f6ecf4c8295ec13bd39502c54 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:36.482350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0da3ad-cbbe-4669-a06d-2748a45a4d6e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:36.482912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0da3ad-cbbe-4669-a06d-2748a45a4d6e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:36.483100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0da3ad-cbbe-4669-a06d-2748a45a4d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:36.506952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5a0da3ad-cbbe-4669-a06d-2748a45a4d6e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:36.508140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5a0da3ad-cbbe-4669-a06d-2748a45a4d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:36.576536 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1322/2635] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:36.589847 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:36.591422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:36.632386 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:36.659144 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f76711-a225-4686-b250-4bb491c4a85b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:36.662990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbeac46-aa45-4d01-8a3a-a0f38070aee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:36.664244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbeac46-aa45-4d01-8a3a-a0f38070aee5 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:36.664520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbeac46-aa45-4d01-8a3a-a0f38070aee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:36.664813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbeac46-aa45-4d01-8a3a-a0f38070aee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:36.682100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbeac46-aa45-4d01-8a3a-a0f38070aee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:36.683634 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbeac46-aa45-4d01-8a3a-a0f38070aee5 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:36.699517 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1314/2636] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:36.986213 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0da3ad-cbbe-4669-a06d-2748a45a4d6e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:36.987725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5a0da3ad-cbbe-4669-a06d-2748a45a4d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:37.002514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0da3ad-cbbe-4669-a06d-2748a45a4d6e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:37.007594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8009ad-2945-4f18-9120-905e7a06e5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:37.008274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8009ad-2945-4f18-9120-905e7a06e5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:37.008525 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8009ad-2945-4f18-9120-905e7a06e5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:37.008831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8009ad-2945-4f18-9120-905e7a06e5eb admin admin] PATH_INFO: `/v3/projects/eb07d4b53c7d44a3b47854f3f0d27b25` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:37.010020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8009ad-2945-4f18-9120-905e7a06e5eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb07d4b53c7d44a3b47854f3f0d27b25)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:37.011923 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8009ad-2945-4f18-9120-905e7a06e5eb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:37.061218 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f8009ad-2945-4f18-9120-905e7a06e5eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb07d4b53c7d44a3b47854f3f0d27b25', 'request_id': 'req-4f8009ad-2945-4f18-9120-905e7a06e5eb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:37.067283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f8009ad-2945-4f18-9120-905e7a06e5eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb07d4b53c7d44a3b47854f3f0d27b25', 'request_id': 'req-4f8009ad-2945-4f18-9120-905e7a06e5eb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:37.073119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f8009ad-2945-4f18-9120-905e7a06e5eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb07d4b53c7d44a3b47854f3f0d27b25', 'request_id': 'req-4f8009ad-2945-4f18-9120-905e7a06e5eb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:37.086373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f8009ad-2945-4f18-9120-905e7a06e5eb admin admin] The token cache is being invalidate because project eb07d4b53c7d44a3b47854f3f0d27b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:37.086654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f8009ad-2945-4f18-9120-905e7a06e5eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:37.087277 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4f8009ad-2945-4f18-9120-905e7a06e5eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:37.088937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1323/2637] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:36 2022] DELETE /identity/v3/projects/eb07d4b53c7d44a3b47854f3f0d27b25 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:37.094104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62979a-a68a-41a1-b585-84a8c6fc6b24 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:37.094472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62979a-a68a-41a1-b585-84a8c6fc6b24 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:37.094811 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62979a-a68a-41a1-b585-84a8c6fc6b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:37.115965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1a62979a-a68a-41a1-b585-84a8c6fc6b24 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:37.117362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1a62979a-a68a-41a1-b585-84a8c6fc6b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:37.179977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1315/2638] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:37.188030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc73d-ef20-4faa-b9b8-f8c7728ef116 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:37.188337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc73d-ef20-4faa-b9b8-f8c7728ef116 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:37.188607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc73d-ef20-4faa-b9b8-f8c7728ef116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:37.209761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d68bc73d-ef20-4faa-b9b8-f8c7728ef116 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:37.210970 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d68bc73d-ef20-4faa-b9b8-f8c7728ef116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:37.219620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1324/2639] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:37.223797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62979a-a68a-41a1-b585-84a8c6fc6b24 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:37.233979 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62979a-a68a-41a1-b585-84a8c6fc6b24 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:37.239086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36590a-97be-4381-9c26-29c6e11182ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:37.239685 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36590a-97be-4381-9c26-29c6e11182ad admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:37.239964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36590a-97be-4381-9c26-29c6e11182ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:37.240233 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36590a-97be-4381-9c26-29c6e11182ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:37.240917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36590a-97be-4381-9c26-29c6e11182ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:25:37.241223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36590a-97be-4381-9c26-29c6e11182ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:37.242846 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36590a-97be-4381-9c26-29c6e11182ad admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:37.254768 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1316/2640] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:25:37.259264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bc73d-ef20-4faa-b9b8-f8c7728ef116 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:37.268919 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bc73d-ef20-4faa-b9b8-f8c7728ef116 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:37.274717 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483ae72-f71c-4a6a-87ea-01fd787d3afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:37.275326 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483ae72-f71c-4a6a-87ea-01fd787d3afa admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:37.275599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483ae72-f71c-4a6a-87ea-01fd787d3afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:37.275862 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483ae72-f71c-4a6a-87ea-01fd787d3afa admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:37.276483 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483ae72-f71c-4a6a-87ea-01fd787d3afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:37.278507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483ae72-f71c-4a6a-87ea-01fd787d3afa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:37.310835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1325/2641] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:37 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:37.315673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36590a-97be-4381-9c26-29c6e11182ad admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:37.326253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36590a-97be-4381-9c26-29c6e11182ad admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:37.391905 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e942432-3c6e-4021-b575-3994e13fc4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:37.392531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e942432-3c6e-4021-b575-3994e13fc4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:37.392838 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e942432-3c6e-4021-b575-3994e13fc4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:37.393131 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e942432-3c6e-4021-b575-3994e13fc4b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:37.393760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e942432-3c6e-4021-b575-3994e13fc4b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:37.396379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e942432-3c6e-4021-b575-3994e13fc4b3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:37.421559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1317/2642] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:37 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:37.426875 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483ae72-f71c-4a6a-87ea-01fd787d3afa admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:37.438568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483ae72-f71c-4a6a-87ea-01fd787d3afa admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:37.443894 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33d6b7-010b-42e5-a85c-758a07eb8b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:37.444501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33d6b7-010b-42e5-a85c-758a07eb8b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:37.444811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33d6b7-010b-42e5-a85c-758a07eb8b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:37.445080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33d6b7-010b-42e5-a85c-758a07eb8b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:37.445748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33d6b7-010b-42e5-a85c-758a07eb8b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:37.447421 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33d6b7-010b-42e5-a85c-758a07eb8b4f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:37.457545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1326/2643] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:37.461677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e942432-3c6e-4021-b575-3994e13fc4b3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:37.472885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e942432-3c6e-4021-b575-3994e13fc4b3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:37.478472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f16d4-1ccf-4201-8fee-7e2a535e27cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:37.479103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f16d4-1ccf-4201-8fee-7e2a535e27cf admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:37.479381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f16d4-1ccf-4201-8fee-7e2a535e27cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:37.479648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f16d4-1ccf-4201-8fee-7e2a535e27cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:37.480294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f16d4-1ccf-4201-8fee-7e2a535e27cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:37.481958 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f16d4-1ccf-4201-8fee-7e2a535e27cf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:37.493253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1318/2644] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:37.497615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33d6b7-010b-42e5-a85c-758a07eb8b4f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:37.509296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33d6b7-010b-42e5-a85c-758a07eb8b4f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:37.514328 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd640d3-bb57-403d-9ce0-6228c0cc14be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:37.514977 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd640d3-bb57-403d-9ce0-6228c0cc14be admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:37.515250 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd640d3-bb57-403d-9ce0-6228c0cc14be admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:37.515516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd640d3-bb57-403d-9ce0-6228c0cc14be admin admin] PATH_INFO: `/v3/projects/8c3a88bdee6e4566987b3e87b4627ca6/users/525ba710ef8b40d0abb27bb2cad1095c/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:37.531364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd640d3-bb57-403d-9ce0-6228c0cc14be admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c3a88bdee6e4566987b3e87b4627ca6, user_id=525ba710ef8b40d0abb27bb2cad1095c, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:37.534095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd640d3-bb57-403d-9ce0-6228c0cc14be admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:37.550051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1327/2645] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:37 2022] PUT /identity/v3/projects/8c3a88bdee6e4566987b3e87b4627ca6/users/525ba710ef8b40d0abb27bb2cad1095c/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:37.555457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11f658-fad6-4908-8a29-9b26f8f27dac None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:37.555747 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11f658-fad6-4908-8a29-9b26f8f27dac None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:37.558614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11f658-fad6-4908-8a29-9b26f8f27dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:37.579289 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-af11f658-fad6-4908-8a29-9b26f8f27dac None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:37.580495 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-af11f658-fad6-4908-8a29-9b26f8f27dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:37.619287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:37.647397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1319/2646] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:37.657567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd640d3-bb57-403d-9ce0-6228c0cc14be admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:37.660051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1fd640d3-bb57-403d-9ce0-6228c0cc14be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:37.682612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:37.734195 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd640d3-bb57-403d-9ce0-6228c0cc14be admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:37.739721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50d0ae-0b67-4f1a-b6b6-26a7cb3611c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:37.740789 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50d0ae-0b67-4f1a-b6b6-26a7cb3611c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:37.741282 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50d0ae-0b67-4f1a-b6b6-26a7cb3611c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:37.741777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50d0ae-0b67-4f1a-b6b6-26a7cb3611c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:37.757857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50d0ae-0b67-4f1a-b6b6-26a7cb3611c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:37.760381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50d0ae-0b67-4f1a-b6b6-26a7cb3611c5 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:37.778317 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1328/2647] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:38.729106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a9cf5-f095-46b9-ac57-ac8a380050aa None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:38.729660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a9cf5-f095-46b9-ac57-ac8a380050aa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:38.729660 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a9cf5-f095-46b9-ac57-ac8a380050aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:38.758571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b65a9cf5-f095-46b9-ac57-ac8a380050aa None None] MFA Rules not processed for user `525ba710ef8b40d0abb27bb2cad1095c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:38.759797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b65a9cf5-f095-46b9-ac57-ac8a380050aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:38.829841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1320/2648] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:38.842893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50d0ae-0b67-4f1a-b6b6-26a7cb3611c5 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:38.845617 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1f50d0ae-0b67-4f1a-b6b6-26a7cb3611c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:38.907696 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50d0ae-0b67-4f1a-b6b6-26a7cb3611c5 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:38.911372 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95202a7b-45d4-4f5e-b295-c33930df8356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:38.911998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95202a7b-45d4-4f5e-b295-c33930df8356 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:38.912302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95202a7b-45d4-4f5e-b295-c33930df8356 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:38.912568 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95202a7b-45d4-4f5e-b295-c33930df8356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:38.922786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95202a7b-45d4-4f5e-b295-c33930df8356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:38.924260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95202a7b-45d4-4f5e-b295-c33930df8356 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:38.934599 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1329/2649] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:40.321363 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f36234-4b1f-47f2-afea-3060aaae6cd9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:40.322005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f36234-4b1f-47f2-afea-3060aaae6cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:40.322210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f36234-4b1f-47f2-afea-3060aaae6cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:40.344601 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f1f36234-4b1f-47f2-afea-3060aaae6cd9 None None] MFA Rules not processed for user `525ba710ef8b40d0abb27bb2cad1095c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:40.346133 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f1f36234-4b1f-47f2-afea-3060aaae6cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:40.356300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1321/2650] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:40.913803 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95202a7b-45d4-4f5e-b295-c33930df8356 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:40.915543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-95202a7b-45d4-4f5e-b295-c33930df8356 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:40.930673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95202a7b-45d4-4f5e-b295-c33930df8356 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:40.935008 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cc176-c71c-4573-98a7-86baddd723a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:40.935684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cc176-c71c-4573-98a7-86baddd723a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:40.935964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cc176-c71c-4573-98a7-86baddd723a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:40.936235 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cc176-c71c-4573-98a7-86baddd723a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:40.939968 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d71cc176-c71c-4573-98a7-86baddd723a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:40.953710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71cc176-c71c-4573-98a7-86baddd723a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:40.955167 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71cc176-c71c-4573-98a7-86baddd723a4 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:40.973092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1330/2651] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:44.903366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f36234-4b1f-47f2-afea-3060aaae6cd9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:44.904889 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f1f36234-4b1f-47f2-afea-3060aaae6cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:44.984585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cc176-c71c-4573-98a7-86baddd723a4 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:44.986746 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d71cc176-c71c-4573-98a7-86baddd723a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:45.008386 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cc176-c71c-4573-98a7-86baddd723a4 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:45.011731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f36234-4b1f-47f2-afea-3060aaae6cd9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:45.013316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e6da3-11f5-4d93-bea4-43d8550399d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:45.013973 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e6da3-11f5-4d93-bea4-43d8550399d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:45.014255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e6da3-11f5-4d93-bea4-43d8550399d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:45.014529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e6da3-11f5-4d93-bea4-43d8550399d1 admin admin] PATH_INFO: `/v3/users/54fe3d99f05c45f099804fccb4679503` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:45.015715 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e6da3-11f5-4d93-bea4-43d8550399d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54fe3d99f05c45f099804fccb4679503)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:45.017247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107b6b0-41ca-483c-8680-47df79f92222 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:45.017738 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e6da3-11f5-4d93-bea4-43d8550399d1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:45.017985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107b6b0-41ca-483c-8680-47df79f92222 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:45.018322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107b6b0-41ca-483c-8680-47df79f92222 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:45.018654 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107b6b0-41ca-483c-8680-47df79f92222 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:45.020422 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c107b6b0-41ca-483c-8680-47df79f92222 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:45.032817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107b6b0-41ca-483c-8680-47df79f92222 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:45.034478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107b6b0-41ca-483c-8680-47df79f92222 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:45.046046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1322/2652] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:45.069611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-392e6da3-11f5-4d93-bea4-43d8550399d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54fe3d99f05c45f099804fccb4679503'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:45.070265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-392e6da3-11f5-4d93-bea4-43d8550399d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54fe3d99f05c45f099804fccb4679503'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:45.079269 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-392e6da3-11f5-4d93-bea4-43d8550399d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54fe3d99f05c45f099804fccb4679503'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:45.091058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-392e6da3-11f5-4d93-bea4-43d8550399d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54fe3d99f05c45f099804fccb4679503'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:45.091728 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-392e6da3-11f5-4d93-bea4-43d8550399d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54fe3d99f05c45f099804fccb4679503'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:45.104877 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1331/2653] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:44 2022] DELETE /identity/v3/users/54fe3d99f05c45f099804fccb4679503 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:45.289320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107b6b0-41ca-483c-8680-47df79f92222 service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:45.306455 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107b6b0-41ca-483c-8680-47df79f92222 service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:45.362820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:45.372926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45255c5-d49f-46bf-8536-42053094004d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:45.373588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45255c5-d49f-46bf-8536-42053094004d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:45.373864 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45255c5-d49f-46bf-8536-42053094004d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:45.374128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45255c5-d49f-46bf-8536-42053094004d admin admin] PATH_INFO: `/v3/projects/2e39683d687e424fbed4539cb4040060` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:45.375272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45255c5-d49f-46bf-8536-42053094004d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e39683d687e424fbed4539cb4040060)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:45.377288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45255c5-d49f-46bf-8536-42053094004d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:45.426350 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d45255c5-d49f-46bf-8536-42053094004d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e39683d687e424fbed4539cb4040060', 'request_id': 'req-d45255c5-d49f-46bf-8536-42053094004d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:45.434716 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d45255c5-d49f-46bf-8536-42053094004d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e39683d687e424fbed4539cb4040060', 'request_id': 'req-d45255c5-d49f-46bf-8536-42053094004d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:45.442810 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d45255c5-d49f-46bf-8536-42053094004d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e39683d687e424fbed4539cb4040060', 'request_id': 'req-d45255c5-d49f-46bf-8536-42053094004d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:45.453938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d45255c5-d49f-46bf-8536-42053094004d admin admin] The token cache is being invalidate because project 2e39683d687e424fbed4539cb4040060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:45.454230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d45255c5-d49f-46bf-8536-42053094004d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:45.454959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d45255c5-d49f-46bf-8536-42053094004d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:45.456554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1323/2654] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:45 2022] DELETE /identity/v3/projects/2e39683d687e424fbed4539cb4040060 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:45.463129 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739ae64-8268-4d65-89b6-66b6a5227479 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:45.463456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739ae64-8268-4d65-89b6-66b6a5227479 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:45.463848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739ae64-8268-4d65-89b6-66b6a5227479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:45.486625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1739ae64-8268-4d65-89b6-66b6a5227479 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:45.487820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1739ae64-8268-4d65-89b6-66b6a5227479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:45.506392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:45.555219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1332/2655] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:45.566234 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8b20-2c9f-4623-8920-c242c0755e30 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:45.566523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8b20-2c9f-4623-8920-c242c0755e30 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:45.566800 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8b20-2c9f-4623-8920-c242c0755e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:45.589151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-53ae8b20-2c9f-4623-8920-c242c0755e30 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:45.590461 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-53ae8b20-2c9f-4623-8920-c242c0755e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:45.599407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1324/2656] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:45.604981 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739ae64-8268-4d65-89b6-66b6a5227479 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:45.626697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739ae64-8268-4d65-89b6-66b6a5227479 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:45.632227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a8d30-7c1b-48e3-9bd5-0a946cba82ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:45.632891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a8d30-7c1b-48e3-9bd5-0a946cba82ca admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:45.633197 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a8d30-7c1b-48e3-9bd5-0a946cba82ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:45.633463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a8d30-7c1b-48e3-9bd5-0a946cba82ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:45.634116 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a8d30-7c1b-48e3-9bd5-0a946cba82ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:25:45.634420 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a8d30-7c1b-48e3-9bd5-0a946cba82ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:45.636046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a8d30-7c1b-48e3-9bd5-0a946cba82ca admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:45.648337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1333/2657] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 21:25:45.653641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae8b20-2c9f-4623-8920-c242c0755e30 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:45.665137 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae8b20-2c9f-4623-8920-c242c0755e30 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:45.671832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821cf94-a4bc-44a7-835b-c00307f5cdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:45.672526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821cf94-a4bc-44a7-835b-c00307f5cdec admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:45.672925 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821cf94-a4bc-44a7-835b-c00307f5cdec admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:45.673161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821cf94-a4bc-44a7-835b-c00307f5cdec admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:45.673821 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821cf94-a4bc-44a7-835b-c00307f5cdec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:45.676013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821cf94-a4bc-44a7-835b-c00307f5cdec admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:45.708259 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1325/2658] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:45 2022] POST /identity/v3/projects => generated 403 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:25:45.712903 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a8d30-7c1b-48e3-9bd5-0a946cba82ca admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:45.723225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a8d30-7c1b-48e3-9bd5-0a946cba82ca admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:45.790419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84411141-1744-40cb-9b14-687408f0b477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:45.791046 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84411141-1744-40cb-9b14-687408f0b477 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:45.791324 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84411141-1744-40cb-9b14-687408f0b477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:45.791592 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84411141-1744-40cb-9b14-687408f0b477 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:45.792247 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84411141-1744-40cb-9b14-687408f0b477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:45.794725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84411141-1744-40cb-9b14-687408f0b477 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:45.819419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1334/2659] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:45 2022] POST /identity/v3/users => generated 302 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:25:45.823973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821cf94-a4bc-44a7-835b-c00307f5cdec admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:45.843507 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821cf94-a4bc-44a7-835b-c00307f5cdec admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:45.848542 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647987d2-03c0-4de3-8f3b-d40f02887a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:45.849172 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647987d2-03c0-4de3-8f3b-d40f02887a35 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:45.849456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647987d2-03c0-4de3-8f3b-d40f02887a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:45.849711 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647987d2-03c0-4de3-8f3b-d40f02887a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:45.850378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647987d2-03c0-4de3-8f3b-d40f02887a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:45.851970 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647987d2-03c0-4de3-8f3b-d40f02887a35 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:45.863042 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1326/2660] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:45.867326 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84411141-1744-40cb-9b14-687408f0b477 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:45.878047 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84411141-1744-40cb-9b14-687408f0b477 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:45.886455 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1a9c1-fb77-4de5-87c9-7d6b89aa3cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:45.887084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1a9c1-fb77-4de5-87c9-7d6b89aa3cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:45.887358 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1a9c1-fb77-4de5-87c9-7d6b89aa3cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:45.887624 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1a9c1-fb77-4de5-87c9-7d6b89aa3cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:45.888268 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1a9c1-fb77-4de5-87c9-7d6b89aa3cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:45.889941 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1a9c1-fb77-4de5-87c9-7d6b89aa3cd8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:45.907556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1335/2661] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:45.912140 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647987d2-03c0-4de3-8f3b-d40f02887a35 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:45.923262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647987d2-03c0-4de3-8f3b-d40f02887a35 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:45.928445 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040afd2d-46cf-4b9e-9a32-2140b66e8386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:45.929153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040afd2d-46cf-4b9e-9a32-2140b66e8386 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:45.929431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040afd2d-46cf-4b9e-9a32-2140b66e8386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:45.929694 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040afd2d-46cf-4b9e-9a32-2140b66e8386 admin admin] PATH_INFO: `/v3/projects/8837ca6ab32344b69ac057ef9dc1d605/users/5bdf563b66d143b5958e65e1bce42ed5/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:45.945914 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040afd2d-46cf-4b9e-9a32-2140b66e8386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8837ca6ab32344b69ac057ef9dc1d605, user_id=5bdf563b66d143b5958e65e1bce42ed5, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:45.948579 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040afd2d-46cf-4b9e-9a32-2140b66e8386 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:45.962365 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1327/2662] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:45 2022] PUT /identity/v3/projects/8837ca6ab32344b69ac057ef9dc1d605/users/5bdf563b66d143b5958e65e1bce42ed5/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:25:45.975224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb9527-e39b-474e-b240-248f5cc8eca4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:45.975614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb9527-e39b-474e-b240-248f5cc8eca4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:45.976012 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb9527-e39b-474e-b240-248f5cc8eca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:46.006378 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e1bb9527-e39b-474e-b240-248f5cc8eca4 None None] MFA Rules not processed for user `5bdf563b66d143b5958e65e1bce42ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:46.007958 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e1bb9527-e39b-474e-b240-248f5cc8eca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:46.078022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1336/2663] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:45 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:46.082827 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040afd2d-46cf-4b9e-9a32-2140b66e8386 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:46.150934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040afd2d-46cf-4b9e-9a32-2140b66e8386 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:46.156483 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032315b5-2d7f-4e7e-9395-6f0de67df0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:46.156483 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032315b5-2d7f-4e7e-9395-6f0de67df0cc admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:46.156757 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032315b5-2d7f-4e7e-9395-6f0de67df0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:46.157015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032315b5-2d7f-4e7e-9395-6f0de67df0cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:46.157634 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032315b5-2d7f-4e7e-9395-6f0de67df0cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:46.159547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032315b5-2d7f-4e7e-9395-6f0de67df0cc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:46.187977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1328/2664] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:46 2022] POST /identity/v3/projects => generated 403 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:46.192590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb9527-e39b-474e-b240-248f5cc8eca4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:46.203870 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb9527-e39b-474e-b240-248f5cc8eca4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:46.220222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:46.266865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f3d17-da6d-499e-9e8e-e8664239b91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:46.267544 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f3d17-da6d-499e-9e8e-e8664239b91c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:46.267829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f3d17-da6d-499e-9e8e-e8664239b91c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:46.268102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f3d17-da6d-499e-9e8e-e8664239b91c admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:46.268743 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f3d17-da6d-499e-9e8e-e8664239b91c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:46.270779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f3d17-da6d-499e-9e8e-e8664239b91c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:46.297665 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1337/2665] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:46 2022] POST /identity/v3/users => generated 308 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:46.302472 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032315b5-2d7f-4e7e-9395-6f0de67df0cc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:46.309966 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:46.314881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032315b5-2d7f-4e7e-9395-6f0de67df0cc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:46.321951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0faa331-1502-4d3e-bf31-a2b5055d71ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:46.322568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0faa331-1502-4d3e-bf31-a2b5055d71ff admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:46.322855 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0faa331-1502-4d3e-bf31-a2b5055d71ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:46.323312 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0faa331-1502-4d3e-bf31-a2b5055d71ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:46.324108 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0faa331-1502-4d3e-bf31-a2b5055d71ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:46.325970 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0faa331-1502-4d3e-bf31-a2b5055d71ff admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:46.337681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1329/2666] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:46.347780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f3d17-da6d-499e-9e8e-e8664239b91c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:46.358994 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f3d17-da6d-499e-9e8e-e8664239b91c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:46.364636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7a30f-b05b-44b6-84c8-5f538c970fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:46.365322 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7a30f-b05b-44b6-84c8-5f538c970fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:46.365609 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7a30f-b05b-44b6-84c8-5f538c970fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:46.365870 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7a30f-b05b-44b6-84c8-5f538c970fd6 admin admin] PATH_INFO: `/v3/projects/f66b68128dfe46c2a1748bad33f9fe17/users/d63fb1943b3d401cb6f726791931502f/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:46.383317 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7a30f-b05b-44b6-84c8-5f538c970fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66b68128dfe46c2a1748bad33f9fe17, user_id=d63fb1943b3d401cb6f726791931502f, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:46.386077 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7a30f-b05b-44b6-84c8-5f538c970fd6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:46.400933 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1338/2667] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:46 2022] PUT /identity/v3/projects/f66b68128dfe46c2a1748bad33f9fe17/users/d63fb1943b3d401cb6f726791931502f/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:46.405125 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0faa331-1502-4d3e-bf31-a2b5055d71ff admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:46.418238 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0faa331-1502-4d3e-bf31-a2b5055d71ff admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:46.481759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7774c-546d-43d8-8955-478cab13a826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:46.482374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7774c-546d-43d8-8955-478cab13a826 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:46.482640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7774c-546d-43d8-8955-478cab13a826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:46.482910 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7774c-546d-43d8-8955-478cab13a826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:46.483562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7774c-546d-43d8-8955-478cab13a826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:46.485251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7774c-546d-43d8-8955-478cab13a826 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:46.495648 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1330/2668] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:46.500168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7a30f-b05b-44b6-84c8-5f538c970fd6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:46.511172 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7a30f-b05b-44b6-84c8-5f538c970fd6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:46.515869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de7546-763e-4539-b4bc-fbb62afe4cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:46.516522 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de7546-763e-4539-b4bc-fbb62afe4cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:46.516839 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de7546-763e-4539-b4bc-fbb62afe4cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:46.517177 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de7546-763e-4539-b4bc-fbb62afe4cb6 admin admin] PATH_INFO: `/v3/projects/f66b68128dfe46c2a1748bad33f9fe17/users/d63fb1943b3d401cb6f726791931502f/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:46.519109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de7546-763e-4539-b4bc-fbb62afe4cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66b68128dfe46c2a1748bad33f9fe17, user_id=d63fb1943b3d401cb6f726791931502f, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:46.521910 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de7546-763e-4539-b4bc-fbb62afe4cb6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:46.533110 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1339/2669] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:46 2022] PUT /identity/v3/projects/f66b68128dfe46c2a1748bad33f9fe17/users/d63fb1943b3d401cb6f726791931502f/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:46.544177 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8676f-b080-47fd-912a-b915c04a70e9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:46.544519 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8676f-b080-47fd-912a-b915c04a70e9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:46.544851 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8676f-b080-47fd-912a-b915c04a70e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:46.576534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2fb8676f-b080-47fd-912a-b915c04a70e9 None None] MFA Rules not processed for user `d63fb1943b3d401cb6f726791931502f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:46.577792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2fb8676f-b080-47fd-912a-b915c04a70e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:46.651441 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1331/2670] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:46 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:46.662614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de7546-763e-4539-b4bc-fbb62afe4cb6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:46.664753 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-74de7546-763e-4539-b4bc-fbb62afe4cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:46.729321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de7546-763e-4539-b4bc-fbb62afe4cb6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:46.733667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7292a4-6a21-4fd4-bc70-ada91f29ed77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:46.734289 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7292a4-6a21-4fd4-bc70-ada91f29ed77 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:46.734566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7292a4-6a21-4fd4-bc70-ada91f29ed77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:46.734831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7292a4-6a21-4fd4-bc70-ada91f29ed77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:46.744843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7292a4-6a21-4fd4-bc70-ada91f29ed77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:46.746301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7292a4-6a21-4fd4-bc70-ada91f29ed77 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:46.756222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1340/2671] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:46 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:46.912535 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8676f-b080-47fd-912a-b915c04a70e9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:46.981359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8676f-b080-47fd-912a-b915c04a70e9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:46.986152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:46.986902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:46.987069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:46.987331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] PATH_INFO: `/v3/users/5bdf563b66d143b5958e65e1bce42ed5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:46.988477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bdf563b66d143b5958e65e1bce42ed5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:46.990495 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:47.008450 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:47.056056 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bdf563b66d143b5958e65e1bce42ed5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.056642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bdf563b66d143b5958e65e1bce42ed5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.064443 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bdf563b66d143b5958e65e1bce42ed5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.083519 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bdf563b66d143b5958e65e1bce42ed5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.084323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bdf563b66d143b5958e65e1bce42ed5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.099410 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1332/2672] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:46 2022] DELETE /identity/v3/users/5bdf563b66d143b5958e65e1bce42ed5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:47.105005 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9d7fa-42c9-4765-84d9-748a79872b7f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:47.105513 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9d7fa-42c9-4765-84d9-748a79872b7f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:47.105942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9d7fa-42c9-4765-84d9-748a79872b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:47.128901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-48d9d7fa-42c9-4765-84d9-748a79872b7f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:47.130253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-48d9d7fa-42c9-4765-84d9-748a79872b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:47.197643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1341/2673] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:47.207881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:47.209532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:47.276446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f76521-4c21-4aa5-8f49-5cb56d0e92b5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:47.280224 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2782d6e-5a07-45eb-87b9-823db37c9e8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:47.280902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2782d6e-5a07-45eb-87b9-823db37c9e8c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:47.281183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2782d6e-5a07-45eb-87b9-823db37c9e8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:47.281457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2782d6e-5a07-45eb-87b9-823db37c9e8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:47.290374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2782d6e-5a07-45eb-87b9-823db37c9e8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:47.291809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2782d6e-5a07-45eb-87b9-823db37c9e8c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:47.302282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1333/2674] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:47.679869 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9d7fa-42c9-4765-84d9-748a79872b7f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:47.681543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-48d9d7fa-42c9-4765-84d9-748a79872b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:47.694151 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9d7fa-42c9-4765-84d9-748a79872b7f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:47.700044 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:47.700741 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:47.701030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:47.701299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] PATH_INFO: `/v3/projects/8837ca6ab32344b69ac057ef9dc1d605` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:47.702712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8837ca6ab32344b69ac057ef9dc1d605)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:47.704673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:47.716527 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:47.761912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8837ca6ab32344b69ac057ef9dc1d605', 'request_id': 'req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.768547 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8837ca6ab32344b69ac057ef9dc1d605', 'request_id': 'req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.774669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8837ca6ab32344b69ac057ef9dc1d605', 'request_id': 'req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.789683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] The token cache is being invalidate because project 8837ca6ab32344b69ac057ef9dc1d605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:47.790082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.790617 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.792444 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1342/2675] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:47 2022] DELETE /identity/v3/projects/8837ca6ab32344b69ac057ef9dc1d605 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:47.797323 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2782d6e-5a07-45eb-87b9-823db37c9e8c service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:47.807486 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2782d6e-5a07-45eb-87b9-823db37c9e8c service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:47.867265 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4718d-0abc-4f0d-a3ee-122ac9f44ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:47.867897 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4718d-0abc-4f0d-a3ee-122ac9f44ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:47.868165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4718d-0abc-4f0d-a3ee-122ac9f44ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:47.868430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4718d-0abc-4f0d-a3ee-122ac9f44ad4 admin admin] PATH_INFO: `/v3/users/d63fb1943b3d401cb6f726791931502f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:47.869639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb4718d-0abc-4f0d-a3ee-122ac9f44ad4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d63fb1943b3d401cb6f726791931502f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:47.871562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb4718d-0abc-4f0d-a3ee-122ac9f44ad4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:47.917024 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:47.924960 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fcb4718d-0abc-4f0d-a3ee-122ac9f44ad4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd63fb1943b3d401cb6f726791931502f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.926231 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fcb4718d-0abc-4f0d-a3ee-122ac9f44ad4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd63fb1943b3d401cb6f726791931502f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.935027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fcb4718d-0abc-4f0d-a3ee-122ac9f44ad4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd63fb1943b3d401cb6f726791931502f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.948798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fcb4718d-0abc-4f0d-a3ee-122ac9f44ad4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd63fb1943b3d401cb6f726791931502f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.949550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fcb4718d-0abc-4f0d-a3ee-122ac9f44ad4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd63fb1943b3d401cb6f726791931502f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:47.964408 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1334/2676] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:47 2022] DELETE /identity/v3/users/d63fb1943b3d401cb6f726791931502f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:48.284190 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:48.284190 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:48.300463 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb0bd2-2345-498f-9cbd-f1e2d14bbda6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:48.358756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df073d2-f8d0-4b80-bf27-bef4b2964890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:48.359430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df073d2-f8d0-4b80-bf27-bef4b2964890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:48.359710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df073d2-f8d0-4b80-bf27-bef4b2964890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:48.360073 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df073d2-f8d0-4b80-bf27-bef4b2964890 admin admin] PATH_INFO: `/v3/projects/f66b68128dfe46c2a1748bad33f9fe17` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:48.361321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df073d2-f8d0-4b80-bf27-bef4b2964890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f66b68128dfe46c2a1748bad33f9fe17)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:48.363319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df073d2-f8d0-4b80-bf27-bef4b2964890 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:48.419056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5df073d2-f8d0-4b80-bf27-bef4b2964890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f66b68128dfe46c2a1748bad33f9fe17', 'request_id': 'req-5df073d2-f8d0-4b80-bf27-bef4b2964890'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:48.425406 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5df073d2-f8d0-4b80-bf27-bef4b2964890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f66b68128dfe46c2a1748bad33f9fe17', 'request_id': 'req-5df073d2-f8d0-4b80-bf27-bef4b2964890'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:48.431906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5df073d2-f8d0-4b80-bf27-bef4b2964890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f66b68128dfe46c2a1748bad33f9fe17', 'request_id': 'req-5df073d2-f8d0-4b80-bf27-bef4b2964890'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:48.446350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5df073d2-f8d0-4b80-bf27-bef4b2964890 admin admin] The token cache is being invalidate because project f66b68128dfe46c2a1748bad33f9fe17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:48.446662 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5df073d2-f8d0-4b80-bf27-bef4b2964890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:48.447473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5df073d2-f8d0-4b80-bf27-bef4b2964890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:48.449468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1343/2677] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:48 2022] DELETE /identity/v3/projects/f66b68128dfe46c2a1748bad33f9fe17 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:48.455301 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faadd51-ed78-48f7-bc01-cff03e54b0f9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:48.455587 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faadd51-ed78-48f7-bc01-cff03e54b0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:48.455859 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faadd51-ed78-48f7-bc01-cff03e54b0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:48.476098 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7faadd51-ed78-48f7-bc01-cff03e54b0f9 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:48.477332 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7faadd51-ed78-48f7-bc01-cff03e54b0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:48.546379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1335/2678] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:48.555982 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fdcf6-1180-40f8-9fff-54d78ea8ade0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:48.556263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fdcf6-1180-40f8-9fff-54d78ea8ade0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:48.556530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fdcf6-1180-40f8-9fff-54d78ea8ade0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:48.577442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e55fdcf6-1180-40f8-9fff-54d78ea8ade0 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:48.578667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e55fdcf6-1180-40f8-9fff-54d78ea8ade0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:48.587545 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1344/2679] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:48.593033 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faadd51-ed78-48f7-bc01-cff03e54b0f9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:48.603158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faadd51-ed78-48f7-bc01-cff03e54b0f9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:48.609337 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221a146-cc51-4837-98ea-4e300684d206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:48.609872 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221a146-cc51-4837-98ea-4e300684d206 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:48.610153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221a146-cc51-4837-98ea-4e300684d206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:48.610409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221a146-cc51-4837-98ea-4e300684d206 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:48.611055 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221a146-cc51-4837-98ea-4e300684d206 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:25:48.611358 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221a146-cc51-4837-98ea-4e300684d206 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:48.613018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221a146-cc51-4837-98ea-4e300684d206 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:48.625971 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1336/2680] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:25:48.630566 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fdcf6-1180-40f8-9fff-54d78ea8ade0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:48.640703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fdcf6-1180-40f8-9fff-54d78ea8ade0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:48.646553 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebc3d4-f9d2-441c-a3f9-5ddd43a0479c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:48.647173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebc3d4-f9d2-441c-a3f9-5ddd43a0479c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:48.647450 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebc3d4-f9d2-441c-a3f9-5ddd43a0479c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:48.647717 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebc3d4-f9d2-441c-a3f9-5ddd43a0479c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:48.648366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ebc3d4-f9d2-441c-a3f9-5ddd43a0479c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:48.650338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ebc3d4-f9d2-441c-a3f9-5ddd43a0479c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:48.673617 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:48.682591 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1345/2681] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:48 2022] POST /identity/v3/projects => generated 393 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:48.686982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221a146-cc51-4837-98ea-4e300684d206 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:48.697538 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221a146-cc51-4837-98ea-4e300684d206 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:48.757828 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0ded0-a939-4906-8329-b906058e7f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:48.758426 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0ded0-a939-4906-8329-b906058e7f6b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:48.758704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0ded0-a939-4906-8329-b906058e7f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:48.758970 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0ded0-a939-4906-8329-b906058e7f6b admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:48.759607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0ded0-a939-4906-8329-b906058e7f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:48.761569 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0ded0-a939-4906-8329-b906058e7f6b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:48.784997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1337/2682] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:48 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:25:48.789526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebc3d4-f9d2-441c-a3f9-5ddd43a0479c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:48.801098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebc3d4-f9d2-441c-a3f9-5ddd43a0479c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:48.806589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a26d6-e309-484e-a09d-c80ad03a2c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:48.807234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a26d6-e309-484e-a09d-c80ad03a2c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:48.807516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a26d6-e309-484e-a09d-c80ad03a2c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:48.807785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a26d6-e309-484e-a09d-c80ad03a2c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:48.808417 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a26d6-e309-484e-a09d-c80ad03a2c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:48.810085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a26d6-e309-484e-a09d-c80ad03a2c85 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:48.820635 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1346/2683] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:48.824841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0ded0-a939-4906-8329-b906058e7f6b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:48.835406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0ded0-a939-4906-8329-b906058e7f6b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:48.842416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850beae-5027-4fd2-b9e6-4a1a77ab2a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:48.843134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850beae-5027-4fd2-b9e6-4a1a77ab2a95 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:48.843409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850beae-5027-4fd2-b9e6-4a1a77ab2a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:48.843670 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850beae-5027-4fd2-b9e6-4a1a77ab2a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:48.844291 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850beae-5027-4fd2-b9e6-4a1a77ab2a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:48.845916 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850beae-5027-4fd2-b9e6-4a1a77ab2a95 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:48.857292 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1338/2684] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:25:48.861792 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a26d6-e309-484e-a09d-c80ad03a2c85 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:48.871643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a26d6-e309-484e-a09d-c80ad03a2c85 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:48.876596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad55d5-d2af-48e3-b2ee-7e3d1ed70fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:48.877274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad55d5-d2af-48e3-b2ee-7e3d1ed70fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:48.877548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad55d5-d2af-48e3-b2ee-7e3d1ed70fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:48.877821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad55d5-d2af-48e3-b2ee-7e3d1ed70fc0 admin admin] PATH_INFO: `/v3/projects/b2f8826108484d73b2e50c6742607e76/users/917bdebafbb647539f11ad484c670e60/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:48.893171 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad55d5-d2af-48e3-b2ee-7e3d1ed70fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2f8826108484d73b2e50c6742607e76, user_id=917bdebafbb647539f11ad484c670e60, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:48.895814 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad55d5-d2af-48e3-b2ee-7e3d1ed70fc0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:48.909204 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1347/2685] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:48 2022] PUT /identity/v3/projects/b2f8826108484d73b2e50c6742607e76/users/917bdebafbb647539f11ad484c670e60/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:48.920122 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384321a-0750-4c5c-8a8f-3fd4479a3ecf None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:48.920694 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384321a-0750-4c5c-8a8f-3fd4479a3ecf None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:48.920694 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384321a-0750-4c5c-8a8f-3fd4479a3ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:48.949784 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8384321a-0750-4c5c-8a8f-3fd4479a3ecf None None] MFA Rules not processed for user `917bdebafbb647539f11ad484c670e60`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:48.950998 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8384321a-0750-4c5c-8a8f-3fd4479a3ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:48.961141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:49.025414 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1339/2686] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:49.031947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad55d5-d2af-48e3-b2ee-7e3d1ed70fc0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:49.101679 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad55d5-d2af-48e3-b2ee-7e3d1ed70fc0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:49.106926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f2d45-8aa0-4fec-9ec2-054bc8094fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:49.107536 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f2d45-8aa0-4fec-9ec2-054bc8094fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:49.107806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f2d45-8aa0-4fec-9ec2-054bc8094fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:49.108114 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f2d45-8aa0-4fec-9ec2-054bc8094fa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:49.108888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f2d45-8aa0-4fec-9ec2-054bc8094fa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:49.110845 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f2d45-8aa0-4fec-9ec2-054bc8094fa3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:49.140538 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1348/2687] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:49 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:49.144990 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384321a-0750-4c5c-8a8f-3fd4479a3ecf None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:49.156903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384321a-0750-4c5c-8a8f-3fd4479a3ecf None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:49.215287 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09069f3-3e51-4672-9208-df852dbdacf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:49.215885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09069f3-3e51-4672-9208-df852dbdacf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:49.216165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09069f3-3e51-4672-9208-df852dbdacf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:49.216433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09069f3-3e51-4672-9208-df852dbdacf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:49.217087 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09069f3-3e51-4672-9208-df852dbdacf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:49.219058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09069f3-3e51-4672-9208-df852dbdacf0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:49.239972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1340/2688] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:49 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:49.244205 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f2d45-8aa0-4fec-9ec2-054bc8094fa3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:49.255095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f2d45-8aa0-4fec-9ec2-054bc8094fa3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:49.264055 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91549c-4df2-4540-9cbf-6d0b267af31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:49.264679 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91549c-4df2-4540-9cbf-6d0b267af31c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:49.265089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91549c-4df2-4540-9cbf-6d0b267af31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:49.265355 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91549c-4df2-4540-9cbf-6d0b267af31c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:49.265995 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91549c-4df2-4540-9cbf-6d0b267af31c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:49.267608 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91549c-4df2-4540-9cbf-6d0b267af31c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:49.278865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1349/2689] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:49.282805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09069f3-3e51-4672-9208-df852dbdacf0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:49.294107 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09069f3-3e51-4672-9208-df852dbdacf0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:49.299609 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6c4d1-cdc6-4baa-8f38-49af016f522a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:49.300281 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6c4d1-cdc6-4baa-8f38-49af016f522a admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:49.300553 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6c4d1-cdc6-4baa-8f38-49af016f522a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:49.300841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6c4d1-cdc6-4baa-8f38-49af016f522a admin admin] PATH_INFO: `/v3/projects/997dbeaf1219409493e4049d5277f011/users/40b33899a7ae42c78a3692f4b0c8f6b3/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:49.321214 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6c4d1-cdc6-4baa-8f38-49af016f522a admin admin] RBAC: Authorizing `identity:create_grant(project_id=997dbeaf1219409493e4049d5277f011, user_id=40b33899a7ae42c78a3692f4b0c8f6b3, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:49.323926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6c4d1-cdc6-4baa-8f38-49af016f522a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:49.339922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1341/2690] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:49 2022] PUT /identity/v3/projects/997dbeaf1219409493e4049d5277f011/users/40b33899a7ae42c78a3692f4b0c8f6b3/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:25:49.344235 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91549c-4df2-4540-9cbf-6d0b267af31c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:49.354831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91549c-4df2-4540-9cbf-6d0b267af31c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:49.422316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998a9f7-7d48-4fee-ad20-1afc81fc7073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:49.423052 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998a9f7-7d48-4fee-ad20-1afc81fc7073 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:49.423396 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998a9f7-7d48-4fee-ad20-1afc81fc7073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:49.423821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998a9f7-7d48-4fee-ad20-1afc81fc7073 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:49.424519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998a9f7-7d48-4fee-ad20-1afc81fc7073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:49.426259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998a9f7-7d48-4fee-ad20-1afc81fc7073 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:49.441846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1350/2691] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:49.443832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6c4d1-cdc6-4baa-8f38-49af016f522a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:49.455811 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6c4d1-cdc6-4baa-8f38-49af016f522a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:49.464061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ff973-887c-430d-9480-e595b8caf979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:49.464061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ff973-887c-430d-9480-e595b8caf979 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:49.466010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ff973-887c-430d-9480-e595b8caf979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:49.466010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ff973-887c-430d-9480-e595b8caf979 admin admin] PATH_INFO: `/v3/projects/997dbeaf1219409493e4049d5277f011/users/40b33899a7ae42c78a3692f4b0c8f6b3/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:49.466449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ff973-887c-430d-9480-e595b8caf979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=997dbeaf1219409493e4049d5277f011, user_id=40b33899a7ae42c78a3692f4b0c8f6b3, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:49.469853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ff973-887c-430d-9480-e595b8caf979 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:49.481045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1342/2692] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:49 2022] PUT /identity/v3/projects/997dbeaf1219409493e4049d5277f011/users/40b33899a7ae42c78a3692f4b0c8f6b3/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:25:49.491673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1013eaf-9587-4a8a-9d47-2808653ac73c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:49.492053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1013eaf-9587-4a8a-9d47-2808653ac73c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:49.492329 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1013eaf-9587-4a8a-9d47-2808653ac73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:49.524408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e1013eaf-9587-4a8a-9d47-2808653ac73c None None] MFA Rules not processed for user `40b33899a7ae42c78a3692f4b0c8f6b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:49.525478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e1013eaf-9587-4a8a-9d47-2808653ac73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:49.549199 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:49.605039 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1351/2693] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:49 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:49.616380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ff973-887c-430d-9480-e595b8caf979 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:49.618190 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d87ff973-887c-430d-9480-e595b8caf979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:49.677530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ff973-887c-430d-9480-e595b8caf979 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:49.681285 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3003a97-2b33-4604-83ce-ae38dfa398f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:49.681895 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3003a97-2b33-4604-83ce-ae38dfa398f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:49.682175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3003a97-2b33-4604-83ce-ae38dfa398f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:49.682442 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3003a97-2b33-4604-83ce-ae38dfa398f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:49.692181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3003a97-2b33-4604-83ce-ae38dfa398f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:49.693793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3003a97-2b33-4604-83ce-ae38dfa398f1 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:49.703881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1343/2694] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:49 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:49.756589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1013eaf-9587-4a8a-9d47-2808653ac73c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:49.758190 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e1013eaf-9587-4a8a-9d47-2808653ac73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:49.771082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1013eaf-9587-4a8a-9d47-2808653ac73c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:49.775267 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c42cb-3d60-4a6e-815b-e27a65a94468 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:49.775909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c42cb-3d60-4a6e-815b-e27a65a94468 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:49.776185 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c42cb-3d60-4a6e-815b-e27a65a94468 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:49.776432 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c42cb-3d60-4a6e-815b-e27a65a94468 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:49.844430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c42cb-3d60-4a6e-815b-e27a65a94468 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:49.845936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c42cb-3d60-4a6e-815b-e27a65a94468 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:49.855365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1352/2695] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:49 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:54.345621 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3003a97-2b33-4604-83ce-ae38dfa398f1 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:54.355522 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:54.415885 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3003a97-2b33-4604-83ce-ae38dfa398f1 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:54.421931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:54.422617 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:54.422894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:54.423145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] PATH_INFO: `/v3/users/917bdebafbb647539f11ad484c670e60` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:54.424284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] RBAC: Authorizing `identity:delete_user(user_id=917bdebafbb647539f11ad484c670e60)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:54.426781 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:54.477646 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '917bdebafbb647539f11ad484c670e60'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:54.478410 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '917bdebafbb647539f11ad484c670e60'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:54.486551 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '917bdebafbb647539f11ad484c670e60'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:54.499961 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '917bdebafbb647539f11ad484c670e60'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:54.500898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '917bdebafbb647539f11ad484c670e60'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:54.517659 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1344/2696] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:54 2022] DELETE /identity/v3/users/917bdebafbb647539f11ad484c670e60 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:54.522423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d320b-067b-4413-a7e0-323aa765d833 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:54.522891 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d320b-067b-4413-a7e0-323aa765d833 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:54.523085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d320b-067b-4413-a7e0-323aa765d833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:54.545113 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-295d320b-067b-4413-a7e0-323aa765d833 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:54.546424 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-295d320b-067b-4413-a7e0-323aa765d833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:54.610069 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1353/2697] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:54.620053 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:54.621626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:54.695945 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bda470-30ee-4933-8546-c04a5704a8af admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:54.702651 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34638f3-af87-4822-a4e3-6145e8b80434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:54.704177 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34638f3-af87-4822-a4e3-6145e8b80434 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:54.704647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34638f3-af87-4822-a4e3-6145e8b80434 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:54.705146 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34638f3-af87-4822-a4e3-6145e8b80434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:54.720591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34638f3-af87-4822-a4e3-6145e8b80434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:54.722104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34638f3-af87-4822-a4e3-6145e8b80434 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:54.732172 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1345/2698] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:55.100051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d320b-067b-4413-a7e0-323aa765d833 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:55.101802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-295d320b-067b-4413-a7e0-323aa765d833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:55.115100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d320b-067b-4413-a7e0-323aa765d833 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:55.120926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:55.121655 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:55.121932 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:55.122213 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] PATH_INFO: `/v3/projects/b2f8826108484d73b2e50c6742607e76` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:55.123452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2f8826108484d73b2e50c6742607e76)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:55.125441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:55.189388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2f8826108484d73b2e50c6742607e76', 'request_id': 'req-e344efd7-e042-468c-a9e2-1fdaaea3783c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.191229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:55.196392 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2f8826108484d73b2e50c6742607e76', 'request_id': 'req-e344efd7-e042-468c-a9e2-1fdaaea3783c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.202702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2f8826108484d73b2e50c6742607e76', 'request_id': 'req-e344efd7-e042-468c-a9e2-1fdaaea3783c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.218183 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] The token cache is being invalidate because project b2f8826108484d73b2e50c6742607e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:55.218472 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.219146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.220685 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1354/2699] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:55 2022] DELETE /identity/v3/projects/b2f8826108484d73b2e50c6742607e76 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:55.224868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34638f3-af87-4822-a4e3-6145e8b80434 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:55.235258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34638f3-af87-4822-a4e3-6145e8b80434 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:55.264004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:55.297135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816288f-3ecd-4cf0-be05-fdd8c00847e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:55.297752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816288f-3ecd-4cf0-be05-fdd8c00847e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:55.298025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816288f-3ecd-4cf0-be05-fdd8c00847e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:55.298290 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816288f-3ecd-4cf0-be05-fdd8c00847e3 admin admin] PATH_INFO: `/v3/users/40b33899a7ae42c78a3692f4b0c8f6b3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:55.299412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816288f-3ecd-4cf0-be05-fdd8c00847e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40b33899a7ae42c78a3692f4b0c8f6b3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:55.301353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816288f-3ecd-4cf0-be05-fdd8c00847e3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:55.360813 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8816288f-3ecd-4cf0-be05-fdd8c00847e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40b33899a7ae42c78a3692f4b0c8f6b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.361683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8816288f-3ecd-4cf0-be05-fdd8c00847e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40b33899a7ae42c78a3692f4b0c8f6b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.370931 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8816288f-3ecd-4cf0-be05-fdd8c00847e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40b33899a7ae42c78a3692f4b0c8f6b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.383599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8816288f-3ecd-4cf0-be05-fdd8c00847e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40b33899a7ae42c78a3692f4b0c8f6b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.384494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8816288f-3ecd-4cf0-be05-fdd8c00847e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40b33899a7ae42c78a3692f4b0c8f6b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.399151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1346/2700] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:55 2022] DELETE /identity/v3/users/40b33899a7ae42c78a3692f4b0c8f6b3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:55.708475 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:55.710546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:55.723562 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344efd7-e042-468c-a9e2-1fdaaea3783c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:55.786084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a708302-1a64-4dcc-bef1-779afa8c7051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:55.786969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a708302-1a64-4dcc-bef1-779afa8c7051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:55.787271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a708302-1a64-4dcc-bef1-779afa8c7051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:55.787546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a708302-1a64-4dcc-bef1-779afa8c7051 admin admin] PATH_INFO: `/v3/projects/997dbeaf1219409493e4049d5277f011` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:55.788988 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a708302-1a64-4dcc-bef1-779afa8c7051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=997dbeaf1219409493e4049d5277f011)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:55.791530 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a708302-1a64-4dcc-bef1-779afa8c7051 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:55.848115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0a708302-1a64-4dcc-bef1-779afa8c7051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '997dbeaf1219409493e4049d5277f011', 'request_id': 'req-0a708302-1a64-4dcc-bef1-779afa8c7051'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.854574 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0a708302-1a64-4dcc-bef1-779afa8c7051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '997dbeaf1219409493e4049d5277f011', 'request_id': 'req-0a708302-1a64-4dcc-bef1-779afa8c7051'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.861456 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0a708302-1a64-4dcc-bef1-779afa8c7051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '997dbeaf1219409493e4049d5277f011', 'request_id': 'req-0a708302-1a64-4dcc-bef1-779afa8c7051'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.877934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0a708302-1a64-4dcc-bef1-779afa8c7051 admin admin] The token cache is being invalidate because project 997dbeaf1219409493e4049d5277f011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:55.878155 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0a708302-1a64-4dcc-bef1-779afa8c7051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.878871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0a708302-1a64-4dcc-bef1-779afa8c7051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:55.880648 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1355/2701] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:55 2022] DELETE /identity/v3/projects/997dbeaf1219409493e4049d5277f011 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:55.886681 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09219cf3-93da-4c6b-a19c-0dbf00572991 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:55.886974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09219cf3-93da-4c6b-a19c-0dbf00572991 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:55.887245 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09219cf3-93da-4c6b-a19c-0dbf00572991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:55.908149 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-09219cf3-93da-4c6b-a19c-0dbf00572991 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:55.909413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-09219cf3-93da-4c6b-a19c-0dbf00572991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:55.972777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1347/2702] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:55.981102 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfefa46-4884-460b-a18f-d31d8eb0ea50 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:55.981385 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfefa46-4884-460b-a18f-d31d8eb0ea50 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:55.981654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfefa46-4884-460b-a18f-d31d8eb0ea50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.003315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1dfefa46-4884-460b-a18f-d31d8eb0ea50 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:56.004569 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1dfefa46-4884-460b-a18f-d31d8eb0ea50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:56.013139 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1356/2703] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:56.017667 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09219cf3-93da-4c6b-a19c-0dbf00572991 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:56.027935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09219cf3-93da-4c6b-a19c-0dbf00572991 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:56.033404 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336f7e4-9d25-4544-a4db-e92edb363cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:56.034020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336f7e4-9d25-4544-a4db-e92edb363cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.034294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336f7e4-9d25-4544-a4db-e92edb363cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.034569 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336f7e4-9d25-4544-a4db-e92edb363cd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.035247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336f7e4-9d25-4544-a4db-e92edb363cd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:25:56.035546 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336f7e4-9d25-4544-a4db-e92edb363cd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:56.037237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336f7e4-9d25-4544-a4db-e92edb363cd6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:56.050223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1348/2704] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:25:56.054987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfefa46-4884-460b-a18f-d31d8eb0ea50 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:56.066964 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfefa46-4884-460b-a18f-d31d8eb0ea50 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:56.072837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0b247-6795-401c-b750-38f705bd074e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:56.073453 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0b247-6795-401c-b750-38f705bd074e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.073724 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0b247-6795-401c-b750-38f705bd074e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.073984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0b247-6795-401c-b750-38f705bd074e admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.074654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0b247-6795-401c-b750-38f705bd074e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:56.076587 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0b247-6795-401c-b750-38f705bd074e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:56.108175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1357/2705] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:56 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:56.113275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336f7e4-9d25-4544-a4db-e92edb363cd6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:56.123858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336f7e4-9d25-4544-a4db-e92edb363cd6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:56.161010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:56.191175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97511b9-3907-497e-93a9-db113bb0a7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:56.191801 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97511b9-3907-497e-93a9-db113bb0a7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.192072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97511b9-3907-497e-93a9-db113bb0a7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.192342 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97511b9-3907-497e-93a9-db113bb0a7c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.193052 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97511b9-3907-497e-93a9-db113bb0a7c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:56.195023 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97511b9-3907-497e-93a9-db113bb0a7c5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:56.217905 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1349/2706] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:56 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:25:56.222264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0b247-6795-401c-b750-38f705bd074e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:56.234919 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0b247-6795-401c-b750-38f705bd074e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:56.240045 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16636cca-8ff0-4319-aeb4-c7427314e3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:56.240681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16636cca-8ff0-4319-aeb4-c7427314e3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.240976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16636cca-8ff0-4319-aeb4-c7427314e3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.241996 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16636cca-8ff0-4319-aeb4-c7427314e3e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.242653 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16636cca-8ff0-4319-aeb4-c7427314e3e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:56.244351 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16636cca-8ff0-4319-aeb4-c7427314e3e8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:56.255779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1358/2707] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:56.260230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97511b9-3907-497e-93a9-db113bb0a7c5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:56.271149 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97511b9-3907-497e-93a9-db113bb0a7c5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:56.276969 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04498c2-aa99-49d6-92c4-cff6603afe15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:56.277583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04498c2-aa99-49d6-92c4-cff6603afe15 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.277855 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04498c2-aa99-49d6-92c4-cff6603afe15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.278123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04498c2-aa99-49d6-92c4-cff6603afe15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.278760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04498c2-aa99-49d6-92c4-cff6603afe15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:56.280460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04498c2-aa99-49d6-92c4-cff6603afe15 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:56.291532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1350/2708] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:56.297020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16636cca-8ff0-4319-aeb4-c7427314e3e8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:56.309690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16636cca-8ff0-4319-aeb4-c7427314e3e8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:56.315403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa387c-1dcf-46e1-9f0c-8ebc1f238b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:56.316089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa387c-1dcf-46e1-9f0c-8ebc1f238b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.316365 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa387c-1dcf-46e1-9f0c-8ebc1f238b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.316636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa387c-1dcf-46e1-9f0c-8ebc1f238b8e admin admin] PATH_INFO: `/v3/projects/c8dd50d664814736b57dfec049a4a2b6/users/a5d6448b40ba42cc9a9643fff44df1b7/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.332631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa387c-1dcf-46e1-9f0c-8ebc1f238b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8dd50d664814736b57dfec049a4a2b6, user_id=a5d6448b40ba42cc9a9643fff44df1b7, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:56.335563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa387c-1dcf-46e1-9f0c-8ebc1f238b8e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:56.354114 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1359/2709] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:56 2022] PUT /identity/v3/projects/c8dd50d664814736b57dfec049a4a2b6/users/a5d6448b40ba42cc9a9643fff44df1b7/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:56.365120 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ef5cf-133a-491b-9e9e-4bd850afb621 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.365699 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ef5cf-133a-491b-9e9e-4bd850afb621 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.365699 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ef5cf-133a-491b-9e9e-4bd850afb621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.396631 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ca3ef5cf-133a-491b-9e9e-4bd850afb621 None None] MFA Rules not processed for user `a5d6448b40ba42cc9a9643fff44df1b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:56.397917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ca3ef5cf-133a-491b-9e9e-4bd850afb621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:56.469251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1351/2710] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:56.474327 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa387c-1dcf-46e1-9f0c-8ebc1f238b8e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:56.485287 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:56.549206 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa387c-1dcf-46e1-9f0c-8ebc1f238b8e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:56.553870 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba89f6-df9e-483b-958a-7cb8f2712ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:56.554499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba89f6-df9e-483b-958a-7cb8f2712ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.554760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba89f6-df9e-483b-958a-7cb8f2712ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.555031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba89f6-df9e-483b-958a-7cb8f2712ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.555670 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba89f6-df9e-483b-958a-7cb8f2712ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:56.557619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba89f6-df9e-483b-958a-7cb8f2712ca7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:56.586984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1360/2711] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:56 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:56.591906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ef5cf-133a-491b-9e9e-4bd850afb621 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:56.603303 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ef5cf-133a-491b-9e9e-4bd850afb621 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:56.676061 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89858c-5664-4598-9113-5e0449a3c430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:56.676683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89858c-5664-4598-9113-5e0449a3c430 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.676933 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89858c-5664-4598-9113-5e0449a3c430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.677226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89858c-5664-4598-9113-5e0449a3c430 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.677880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89858c-5664-4598-9113-5e0449a3c430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:56.679787 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89858c-5664-4598-9113-5e0449a3c430 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:56.703225 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1352/2712] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:56 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:56.707347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba89f6-df9e-483b-958a-7cb8f2712ca7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:56.718596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba89f6-df9e-483b-958a-7cb8f2712ca7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:56.723731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54652f-9f75-481a-a257-1d95c639fc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:56.724395 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54652f-9f75-481a-a257-1d95c639fc83 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.724764 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54652f-9f75-481a-a257-1d95c639fc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.725021 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54652f-9f75-481a-a257-1d95c639fc83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.725673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54652f-9f75-481a-a257-1d95c639fc83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:56.727334 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54652f-9f75-481a-a257-1d95c639fc83 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:56.738945 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1361/2713] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:56.743573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89858c-5664-4598-9113-5e0449a3c430 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:56.755040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89858c-5664-4598-9113-5e0449a3c430 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:56.762399 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4309b40-a237-4f96-8d45-5d80e46645b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:56.763100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4309b40-a237-4f96-8d45-5d80e46645b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.763377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4309b40-a237-4f96-8d45-5d80e46645b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.763646 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4309b40-a237-4f96-8d45-5d80e46645b8 admin admin] PATH_INFO: `/v3/projects/5a201c40c419413db6fec9b25f744c42/users/91dcb87a571a49ff8fff6e947fd5526d/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.780413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4309b40-a237-4f96-8d45-5d80e46645b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a201c40c419413db6fec9b25f744c42, user_id=91dcb87a571a49ff8fff6e947fd5526d, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:56.783314 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4309b40-a237-4f96-8d45-5d80e46645b8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:56.800684 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1353/2714] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:56 2022] PUT /identity/v3/projects/5a201c40c419413db6fec9b25f744c42/users/91dcb87a571a49ff8fff6e947fd5526d/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:56.804788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54652f-9f75-481a-a257-1d95c639fc83 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:56.816428 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54652f-9f75-481a-a257-1d95c639fc83 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:56.878597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1190fcd-271a-4c11-98a5-eb2a6a736eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:56.879409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1190fcd-271a-4c11-98a5-eb2a6a736eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.879787 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1190fcd-271a-4c11-98a5-eb2a6a736eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.880072 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1190fcd-271a-4c11-98a5-eb2a6a736eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.880868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1190fcd-271a-4c11-98a5-eb2a6a736eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:56.882630 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1190fcd-271a-4c11-98a5-eb2a6a736eb1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:56.893551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1362/2715] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:56.898130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4309b40-a237-4f96-8d45-5d80e46645b8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:56.909652 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4309b40-a237-4f96-8d45-5d80e46645b8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:56.914714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae242eab-6b1f-4f1f-a763-eeaa2f54ea6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:56.915378 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae242eab-6b1f-4f1f-a763-eeaa2f54ea6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.915657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae242eab-6b1f-4f1f-a763-eeaa2f54ea6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.915932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae242eab-6b1f-4f1f-a763-eeaa2f54ea6b admin admin] PATH_INFO: `/v3/projects/5a201c40c419413db6fec9b25f744c42/users/91dcb87a571a49ff8fff6e947fd5526d/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.918237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae242eab-6b1f-4f1f-a763-eeaa2f54ea6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a201c40c419413db6fec9b25f744c42, user_id=91dcb87a571a49ff8fff6e947fd5526d, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:56.920988 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae242eab-6b1f-4f1f-a763-eeaa2f54ea6b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:56.924329 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:56.933904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1354/2716] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:25:56 2022] PUT /identity/v3/projects/5a201c40c419413db6fec9b25f744c42/users/91dcb87a571a49ff8fff6e947fd5526d/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:56.945496 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bb518-f28c-4fd9-ac80-77c2e2deadf1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:56.945770 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bb518-f28c-4fd9-ac80-77c2e2deadf1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:56.946050 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bb518-f28c-4fd9-ac80-77c2e2deadf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:56.978037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e97bb518-f28c-4fd9-ac80-77c2e2deadf1 None None] MFA Rules not processed for user `91dcb87a571a49ff8fff6e947fd5526d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:56.979246 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e97bb518-f28c-4fd9-ac80-77c2e2deadf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:57.056283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1363/2717] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:57.068490 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae242eab-6b1f-4f1f-a763-eeaa2f54ea6b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:57.070594 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ae242eab-6b1f-4f1f-a763-eeaa2f54ea6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:57.145167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae242eab-6b1f-4f1f-a763-eeaa2f54ea6b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:57.151677 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23ea45-0425-42a3-8518-6b7a1e6338c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:57.152426 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23ea45-0425-42a3-8518-6b7a1e6338c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:57.152941 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23ea45-0425-42a3-8518-6b7a1e6338c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:57.153416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23ea45-0425-42a3-8518-6b7a1e6338c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:57.168603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f23ea45-0425-42a3-8518-6b7a1e6338c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:57.170237 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f23ea45-0425-42a3-8518-6b7a1e6338c0 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:57.182708 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1355/2718] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:57.870584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bb518-f28c-4fd9-ac80-77c2e2deadf1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:57.875703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:57.949548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bb518-f28c-4fd9-ac80-77c2e2deadf1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:57.956604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:57.957710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:57.957710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:57.957710 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] PATH_INFO: `/v3/users/a5d6448b40ba42cc9a9643fff44df1b7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:57.959644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d6448b40ba42cc9a9643fff44df1b7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:57.961677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:58.015423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d6448b40ba42cc9a9643fff44df1b7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.016096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d6448b40ba42cc9a9643fff44df1b7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.024032 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d6448b40ba42cc9a9643fff44df1b7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.037511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d6448b40ba42cc9a9643fff44df1b7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.038276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d6448b40ba42cc9a9643fff44df1b7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.055136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1364/2719] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:57 2022] DELETE /identity/v3/users/a5d6448b40ba42cc9a9643fff44df1b7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:58.061400 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e2416-9ebc-47ff-af2e-06fbbf77054f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:58.061902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e2416-9ebc-47ff-af2e-06fbbf77054f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:58.062005 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e2416-9ebc-47ff-af2e-06fbbf77054f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:58.081945 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-216e2416-9ebc-47ff-af2e-06fbbf77054f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:58.083263 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-216e2416-9ebc-47ff-af2e-06fbbf77054f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:58.148189 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1356/2720] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:25:58.159729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:58.161233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:58.223734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32630b6-cb12-4a34-8345-2104f52d3b79 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:58.227918 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41499383-39a6-421d-954d-058d6422c57c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:58.228539 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41499383-39a6-421d-954d-058d6422c57c service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:58.228878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41499383-39a6-421d-954d-058d6422c57c service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:58.229146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41499383-39a6-421d-954d-058d6422c57c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:58.239503 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41499383-39a6-421d-954d-058d6422c57c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:58.241061 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41499383-39a6-421d-954d-058d6422c57c service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:58.251234 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1365/2721] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:25:58.532451 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e494ba-580e-45a3-8fed-0b6f6417d8b5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:58.532883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e494ba-580e-45a3-8fed-0b6f6417d8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:58.533247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e494ba-580e-45a3-8fed-0b6f6417d8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:58.556457 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-71e494ba-580e-45a3-8fed-0b6f6417d8b5 None None] MFA Rules not processed for user `525ba710ef8b40d0abb27bb2cad1095c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:58.557726 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-71e494ba-580e-45a3-8fed-0b6f6417d8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:58.562749 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41499383-39a6-421d-954d-058d6422c57c service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:58.565331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-41499383-39a6-421d-954d-058d6422c57c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:58.583437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41499383-39a6-421d-954d-058d6422c57c service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:58.594821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:58.596142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:58.596492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:58.597039 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] PATH_INFO: `/v3/projects/c8dd50d664814736b57dfec049a4a2b6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:58.599682 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8dd50d664814736b57dfec049a4a2b6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:58.602106 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:58.629415 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:58.634367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1357/2722] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:58.660707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8dd50d664814736b57dfec049a4a2b6', 'request_id': 'req-49772d41-bd43-4508-ad39-0612c558677f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.669577 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8dd50d664814736b57dfec049a4a2b6', 'request_id': 'req-49772d41-bd43-4508-ad39-0612c558677f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.675577 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8dd50d664814736b57dfec049a4a2b6', 'request_id': 'req-49772d41-bd43-4508-ad39-0612c558677f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.692884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] The token cache is being invalidate because project c8dd50d664814736b57dfec049a4a2b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:58.693302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.694100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.695733 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1366/2723] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:58 2022] DELETE /identity/v3/projects/c8dd50d664814736b57dfec049a4a2b6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:58.699959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e494ba-580e-45a3-8fed-0b6f6417d8b5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:58.712222 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e494ba-580e-45a3-8fed-0b6f6417d8b5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:58.759911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:58.777904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e1eea-2964-4bc3-a040-f65ba0aae5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:58.778454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e1eea-2964-4bc3-a040-f65ba0aae5b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:58.778728 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e1eea-2964-4bc3-a040-f65ba0aae5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:58.778995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e1eea-2964-4bc3-a040-f65ba0aae5b1 admin admin] PATH_INFO: `/v3/users/91dcb87a571a49ff8fff6e947fd5526d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:58.780197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e1eea-2964-4bc3-a040-f65ba0aae5b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91dcb87a571a49ff8fff6e947fd5526d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:58.782183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e1eea-2964-4bc3-a040-f65ba0aae5b1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:58.852558 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-db2e1eea-2964-4bc3-a040-f65ba0aae5b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91dcb87a571a49ff8fff6e947fd5526d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.853416 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-db2e1eea-2964-4bc3-a040-f65ba0aae5b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91dcb87a571a49ff8fff6e947fd5526d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.861015 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-db2e1eea-2964-4bc3-a040-f65ba0aae5b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91dcb87a571a49ff8fff6e947fd5526d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.874544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-db2e1eea-2964-4bc3-a040-f65ba0aae5b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91dcb87a571a49ff8fff6e947fd5526d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.875350 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-db2e1eea-2964-4bc3-a040-f65ba0aae5b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91dcb87a571a49ff8fff6e947fd5526d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:58.890671 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1358/2724] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:25:58 2022] DELETE /identity/v3/users/91dcb87a571a49ff8fff6e947fd5526d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:59.331056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:59.332785 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:59.363804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49772d41-bd43-4508-ad39-0612c558677f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:59.450459 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817d106-5d48-41c5-a398-603900c43ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:59.451117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817d106-5d48-41c5-a398-603900c43ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:59.451386 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817d106-5d48-41c5-a398-603900c43ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:59.451731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817d106-5d48-41c5-a398-603900c43ccc admin admin] PATH_INFO: `/v3/projects/5a201c40c419413db6fec9b25f744c42` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:59.452913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817d106-5d48-41c5-a398-603900c43ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a201c40c419413db6fec9b25f744c42)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:59.454889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817d106-5d48-41c5-a398-603900c43ccc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:59.508805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b817d106-5d48-41c5-a398-603900c43ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a201c40c419413db6fec9b25f744c42', 'request_id': 'req-b817d106-5d48-41c5-a398-603900c43ccc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:59.516337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b817d106-5d48-41c5-a398-603900c43ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a201c40c419413db6fec9b25f744c42', 'request_id': 'req-b817d106-5d48-41c5-a398-603900c43ccc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:59.523776 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b817d106-5d48-41c5-a398-603900c43ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a201c40c419413db6fec9b25f744c42', 'request_id': 'req-b817d106-5d48-41c5-a398-603900c43ccc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:59.540205 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b817d106-5d48-41c5-a398-603900c43ccc admin admin] The token cache is being invalidate because project 5a201c40c419413db6fec9b25f744c42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:25:59.540537 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b817d106-5d48-41c5-a398-603900c43ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:59.541226 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b817d106-5d48-41c5-a398-603900c43ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:25:59.543271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1367/2725] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:25:59 2022] DELETE /identity/v3/projects/5a201c40c419413db6fec9b25f744c42 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:25:59.548150 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87486f-020a-465b-9a5a-39fa1b776a45 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:59.548741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87486f-020a-465b-9a5a-39fa1b776a45 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:59.549091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87486f-020a-465b-9a5a-39fa1b776a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:59.574310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3c87486f-020a-465b-9a5a-39fa1b776a45 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:59.575556 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3c87486f-020a-465b-9a5a-39fa1b776a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:59.669763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1359/2726] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:59.677864 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3104c7-45d4-4b92-91c1-15c56e9bb73e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:59.678161 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3104c7-45d4-4b92-91c1-15c56e9bb73e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:59.678441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3104c7-45d4-4b92-91c1-15c56e9bb73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:59.700239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4e3104c7-45d4-4b92-91c1-15c56e9bb73e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:25:59.701519 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4e3104c7-45d4-4b92-91c1-15c56e9bb73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:25:59.713898 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1368/2727] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:25:59.715418 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87486f-020a-465b-9a5a-39fa1b776a45 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:59.727835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87486f-020a-465b-9a5a-39fa1b776a45 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:59.733091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41242eb-e671-4e07-863c-7f2534929c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:59.733814 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41242eb-e671-4e07-863c-7f2534929c44 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:59.734157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41242eb-e671-4e07-863c-7f2534929c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:59.734518 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41242eb-e671-4e07-863c-7f2534929c44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:59.735277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41242eb-e671-4e07-863c-7f2534929c44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:25:59.735640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41242eb-e671-4e07-863c-7f2534929c44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:59.737313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41242eb-e671-4e07-863c-7f2534929c44 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:59.751320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1360/2728] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:25:59.756137 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3104c7-45d4-4b92-91c1-15c56e9bb73e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:59.766310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3104c7-45d4-4b92-91c1-15c56e9bb73e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:59.773306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc551-46df-454b-b6f3-60e54d93e7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:59.773937 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc551-46df-454b-b6f3-60e54d93e7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:59.774209 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc551-46df-454b-b6f3-60e54d93e7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:59.774474 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc551-46df-454b-b6f3-60e54d93e7a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:59.775121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bc551-46df-454b-b6f3-60e54d93e7a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:59.777121 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bc551-46df-454b-b6f3-60e54d93e7a6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:59.811598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1369/2729] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:25:59 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:59.815982 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41242eb-e671-4e07-863c-7f2534929c44 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:59.827890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41242eb-e671-4e07-863c-7f2534929c44 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:59.884953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:59.898359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15467d-2794-4169-9804-0c8f2c358874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:59.898777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15467d-2794-4169-9804-0c8f2c358874 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:59.899091 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15467d-2794-4169-9804-0c8f2c358874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:59.899360 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15467d-2794-4169-9804-0c8f2c358874 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:59.900050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15467d-2794-4169-9804-0c8f2c358874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:59.902464 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15467d-2794-4169-9804-0c8f2c358874 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:59.931689 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1361/2730] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:25:59 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:25:59.936103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc551-46df-454b-b6f3-60e54d93e7a6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:25:59.947707 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc551-46df-454b-b6f3-60e54d93e7a6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:25:59.952728 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7770b4f-cb88-47fb-b259-a1e8c96e51b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:25:59.953347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7770b4f-cb88-47fb-b259-a1e8c96e51b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:25:59.953620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7770b4f-cb88-47fb-b259-a1e8c96e51b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:25:59.953887 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7770b4f-cb88-47fb-b259-a1e8c96e51b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:25:59.954527 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7770b4f-cb88-47fb-b259-a1e8c96e51b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:25:59.956152 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7770b4f-cb88-47fb-b259-a1e8c96e51b9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:25:59.958474 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:25:59.985368 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1370/2731] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:25:59.991181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15467d-2794-4169-9804-0c8f2c358874 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:00.002479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15467d-2794-4169-9804-0c8f2c358874 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:00.008318 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea040ca9-6282-4532-901f-09997e0047fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:00.009064 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea040ca9-6282-4532-901f-09997e0047fd admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:00.009340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea040ca9-6282-4532-901f-09997e0047fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:00.009942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea040ca9-6282-4532-901f-09997e0047fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:00.010470 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea040ca9-6282-4532-901f-09997e0047fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:00.012356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea040ca9-6282-4532-901f-09997e0047fd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:00.024718 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1362/2732] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:26:00.029022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7770b4f-cb88-47fb-b259-a1e8c96e51b9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:00.041533 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7770b4f-cb88-47fb-b259-a1e8c96e51b9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:00.047831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce435251-3063-4854-8bde-d243d98ddaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:00.048520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce435251-3063-4854-8bde-d243d98ddaf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:00.048837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce435251-3063-4854-8bde-d243d98ddaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:00.049112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce435251-3063-4854-8bde-d243d98ddaf9 admin admin] PATH_INFO: `/v3/projects/10cfd3874d7c4b07bc55c46ee6953cb5/users/7e82e304a4be4cdbb14a402cb9a90ad9/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:00.066346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce435251-3063-4854-8bde-d243d98ddaf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10cfd3874d7c4b07bc55c46ee6953cb5, user_id=7e82e304a4be4cdbb14a402cb9a90ad9, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:00.069539 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce435251-3063-4854-8bde-d243d98ddaf9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:00.086111 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1371/2733] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:26:00 2022] PUT /identity/v3/projects/10cfd3874d7c4b07bc55c46ee6953cb5/users/7e82e304a4be4cdbb14a402cb9a90ad9/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:26:00.096502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad80f47-7287-4f1c-8b04-0bc358265bb8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:00.096848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad80f47-7287-4f1c-8b04-0bc358265bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:00.097121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad80f47-7287-4f1c-8b04-0bc358265bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:00.129820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0ad80f47-7287-4f1c-8b04-0bc358265bb8 None None] MFA Rules not processed for user `7e82e304a4be4cdbb14a402cb9a90ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:26:00.131060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0ad80f47-7287-4f1c-8b04-0bc358265bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:00.200546 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1363/2734] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:26:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:26:00.205537 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce435251-3063-4854-8bde-d243d98ddaf9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:00.272461 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce435251-3063-4854-8bde-d243d98ddaf9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:00.277491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98805caf-87cf-4365-b55a-09b39dcdf155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:00.278097 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98805caf-87cf-4365-b55a-09b39dcdf155 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:00.278374 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98805caf-87cf-4365-b55a-09b39dcdf155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:00.278642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98805caf-87cf-4365-b55a-09b39dcdf155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:00.279284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98805caf-87cf-4365-b55a-09b39dcdf155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:00.281261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98805caf-87cf-4365-b55a-09b39dcdf155 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:00.311222 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1372/2735] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:26:00 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:26:00.315520 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad80f47-7287-4f1c-8b04-0bc358265bb8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:00.327210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad80f47-7287-4f1c-8b04-0bc358265bb8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:00.388240 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299651f-b5c4-45ec-8022-124bbb2d3163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:00.388922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299651f-b5c4-45ec-8022-124bbb2d3163 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:00.389200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299651f-b5c4-45ec-8022-124bbb2d3163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:00.389505 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299651f-b5c4-45ec-8022-124bbb2d3163 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:00.390147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299651f-b5c4-45ec-8022-124bbb2d3163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:00.392083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299651f-b5c4-45ec-8022-124bbb2d3163 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:00.418008 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1364/2736] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:26:00 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:26:00.423169 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98805caf-87cf-4365-b55a-09b39dcdf155 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:00.434148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98805caf-87cf-4365-b55a-09b39dcdf155 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:00.440509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac849f-5af2-4903-9059-56c384f47690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:00.441175 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac849f-5af2-4903-9059-56c384f47690 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:00.441567 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac849f-5af2-4903-9059-56c384f47690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:00.442002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac849f-5af2-4903-9059-56c384f47690 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:00.442657 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac849f-5af2-4903-9059-56c384f47690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:00.444291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac849f-5af2-4903-9059-56c384f47690 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:00.456731 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1373/2737] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:26:00.461403 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299651f-b5c4-45ec-8022-124bbb2d3163 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:00.476830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299651f-b5c4-45ec-8022-124bbb2d3163 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:00.483074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ebaca-def1-4f87-aa07-bcd23279c6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:00.484071 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ebaca-def1-4f87-aa07-bcd23279c6ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:00.484423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ebaca-def1-4f87-aa07-bcd23279c6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:00.484775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ebaca-def1-4f87-aa07-bcd23279c6ef admin admin] PATH_INFO: `/v3/projects/c0af49a96daf40429b1952d158e36f6a/users/791bd6af720746ad91f288a1e6f17e4e/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:00.505819 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ebaca-def1-4f87-aa07-bcd23279c6ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0af49a96daf40429b1952d158e36f6a, user_id=791bd6af720746ad91f288a1e6f17e4e, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:00.508478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ebaca-def1-4f87-aa07-bcd23279c6ef admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:00.535934 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1365/2738] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:26:00 2022] PUT /identity/v3/projects/c0af49a96daf40429b1952d158e36f6a/users/791bd6af720746ad91f288a1e6f17e4e/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:26:00.536153 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac849f-5af2-4903-9059-56c384f47690 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:00.540820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac849f-5af2-4903-9059-56c384f47690 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:00.607861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774858a-131a-4af3-93ed-aada91f685f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:00.608550 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774858a-131a-4af3-93ed-aada91f685f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:00.608858 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774858a-131a-4af3-93ed-aada91f685f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:00.609131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774858a-131a-4af3-93ed-aada91f685f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:00.609778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774858a-131a-4af3-93ed-aada91f685f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:00.611398 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774858a-131a-4af3-93ed-aada91f685f5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:00.622338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1374/2739] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:26:00.627128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ebaca-def1-4f87-aa07-bcd23279c6ef admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:00.638492 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ebaca-def1-4f87-aa07-bcd23279c6ef admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:00.643869 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559bc73-9880-4835-9b1d-11acfc525501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:00.644683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559bc73-9880-4835-9b1d-11acfc525501 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:00.644974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559bc73-9880-4835-9b1d-11acfc525501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:00.645256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559bc73-9880-4835-9b1d-11acfc525501 admin admin] PATH_INFO: `/v3/projects/c0af49a96daf40429b1952d158e36f6a/users/791bd6af720746ad91f288a1e6f17e4e/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:00.647316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559bc73-9880-4835-9b1d-11acfc525501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0af49a96daf40429b1952d158e36f6a, user_id=791bd6af720746ad91f288a1e6f17e4e, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:00.650200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559bc73-9880-4835-9b1d-11acfc525501 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:00.661841 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1366/2740] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:26:00 2022] PUT /identity/v3/projects/c0af49a96daf40429b1952d158e36f6a/users/791bd6af720746ad91f288a1e6f17e4e/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:26:00.674993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a905a-daa9-41f8-8ca5-2ec6aa72a051 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:00.675274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a905a-daa9-41f8-8ca5-2ec6aa72a051 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:00.675509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a905a-daa9-41f8-8ca5-2ec6aa72a051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:00.707672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b99a905a-daa9-41f8-8ca5-2ec6aa72a051 None None] MFA Rules not processed for user `791bd6af720746ad91f288a1e6f17e4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:26:00.708991 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b99a905a-daa9-41f8-8ca5-2ec6aa72a051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:00.743051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:26:00.783447 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1375/2741] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:26:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:26:00.794977 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559bc73-9880-4835-9b1d-11acfc525501 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:00.797225 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6559bc73-9880-4835-9b1d-11acfc525501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:00.814700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:26:00.871549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559bc73-9880-4835-9b1d-11acfc525501 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:00.876161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a35df-4133-47ba-9881-7ad8c7b1d42e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:00.876912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a35df-4133-47ba-9881-7ad8c7b1d42e service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:00.877330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a35df-4133-47ba-9881-7ad8c7b1d42e service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:00.877666 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a35df-4133-47ba-9881-7ad8c7b1d42e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:00.980830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a35df-4133-47ba-9881-7ad8c7b1d42e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:00.982967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a35df-4133-47ba-9881-7ad8c7b1d42e service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:00.999511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1367/2742] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:26:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:26:05.469581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a905a-daa9-41f8-8ca5-2ec6aa72a051 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:05.471051 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b99a905a-daa9-41f8-8ca5-2ec6aa72a051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:05.484418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a905a-daa9-41f8-8ca5-2ec6aa72a051 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:05.488404 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679fbb5-d71d-44f4-9cf9-93d5cb93fbe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:05.489169 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679fbb5-d71d-44f4-9cf9-93d5cb93fbe9 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:05.489447 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679fbb5-d71d-44f4-9cf9-93d5cb93fbe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:05.489773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679fbb5-d71d-44f4-9cf9-93d5cb93fbe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:05.499920 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9679fbb5-d71d-44f4-9cf9-93d5cb93fbe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:05.501402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9679fbb5-d71d-44f4-9cf9-93d5cb93fbe9 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:05.511071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1376/2743] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:26:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:26:10.606174 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a35df-4133-47ba-9881-7ad8c7b1d42e service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:10.673468 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a35df-4133-47ba-9881-7ad8c7b1d42e service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:10.678777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:10.679418 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:10.679695 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:10.679992 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] PATH_INFO: `/v3/users/7e82e304a4be4cdbb14a402cb9a90ad9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:10.681283 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e82e304a4be4cdbb14a402cb9a90ad9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:10.683218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:10.733672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e82e304a4be4cdbb14a402cb9a90ad9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:10.734306 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e82e304a4be4cdbb14a402cb9a90ad9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:10.741339 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e82e304a4be4cdbb14a402cb9a90ad9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:10.749161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:26:10.754600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e82e304a4be4cdbb14a402cb9a90ad9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:10.755389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e82e304a4be4cdbb14a402cb9a90ad9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:10.768456 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1368/2744] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:26:10 2022] DELETE /identity/v3/users/7e82e304a4be4cdbb14a402cb9a90ad9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:26:10.773203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac491c4-c796-457b-9505-4de27ef08fe8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:10.773715 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac491c4-c796-457b-9505-4de27ef08fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:10.773715 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac491c4-c796-457b-9505-4de27ef08fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:10.793841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5ac491c4-c796-457b-9505-4de27ef08fe8 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:26:10.795064 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5ac491c4-c796-457b-9505-4de27ef08fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:10.856471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1377/2745] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:26:10.866752 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:10.868433 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:10.930767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40506311-e2f1-4621-a91e-373feccc35af admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:10.934806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601202e-d2fa-4113-8c96-58c02d95f2b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:10.935418 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601202e-d2fa-4113-8c96-58c02d95f2b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:10.935698 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601202e-d2fa-4113-8c96-58c02d95f2b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:10.936047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601202e-d2fa-4113-8c96-58c02d95f2b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:10.945543 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601202e-d2fa-4113-8c96-58c02d95f2b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:10.946985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601202e-d2fa-4113-8c96-58c02d95f2b9 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:10.957029 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1369/2746] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:26:11.243369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac491c4-c796-457b-9505-4de27ef08fe8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:11.245053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5ac491c4-c796-457b-9505-4de27ef08fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:11.258298 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac491c4-c796-457b-9505-4de27ef08fe8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:11.269074 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:11.270040 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:11.270418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:11.270786 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] PATH_INFO: `/v3/projects/10cfd3874d7c4b07bc55c46ee6953cb5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:11.272613 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10cfd3874d7c4b07bc55c46ee6953cb5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:11.279799 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:11.329620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10cfd3874d7c4b07bc55c46ee6953cb5', 'request_id': 'req-55f8ccd0-5758-4ca3-a207-482c197152a2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:11.336464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10cfd3874d7c4b07bc55c46ee6953cb5', 'request_id': 'req-55f8ccd0-5758-4ca3-a207-482c197152a2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:11.342313 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10cfd3874d7c4b07bc55c46ee6953cb5', 'request_id': 'req-55f8ccd0-5758-4ca3-a207-482c197152a2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:11.354118 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] The token cache is being invalidate because project 10cfd3874d7c4b07bc55c46ee6953cb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:26:11.354357 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:11.355011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:11.356537 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1378/2747] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:26:11 2022] DELETE /identity/v3/projects/10cfd3874d7c4b07bc55c46ee6953cb5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:26:11.361155 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601202e-d2fa-4113-8c96-58c02d95f2b9 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:11.371059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601202e-d2fa-4113-8c96-58c02d95f2b9 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:11.434559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62202f-ce5d-410e-9695-064a6f9c4f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:11.435192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62202f-ce5d-410e-9695-064a6f9c4f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:11.435565 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62202f-ce5d-410e-9695-064a6f9c4f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:11.435824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62202f-ce5d-410e-9695-064a6f9c4f31 admin admin] PATH_INFO: `/v3/users/791bd6af720746ad91f288a1e6f17e4e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:11.437319 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62202f-ce5d-410e-9695-064a6f9c4f31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=791bd6af720746ad91f288a1e6f17e4e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:11.439277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62202f-ce5d-410e-9695-064a6f9c4f31 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:11.490515 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cc62202f-ce5d-410e-9695-064a6f9c4f31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '791bd6af720746ad91f288a1e6f17e4e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:11.491207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cc62202f-ce5d-410e-9695-064a6f9c4f31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '791bd6af720746ad91f288a1e6f17e4e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:11.498184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cc62202f-ce5d-410e-9695-064a6f9c4f31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '791bd6af720746ad91f288a1e6f17e4e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:11.510203 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cc62202f-ce5d-410e-9695-064a6f9c4f31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '791bd6af720746ad91f288a1e6f17e4e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:11.510912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cc62202f-ce5d-410e-9695-064a6f9c4f31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '791bd6af720746ad91f288a1e6f17e4e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:11.528313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1370/2748] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:26:11 2022] DELETE /identity/v3/users/791bd6af720746ad91f288a1e6f17e4e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:26:11.924164 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:11.925587 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:11.942164 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8ccd0-5758-4ca3-a207-482c197152a2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:11.992694 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:26:12.013740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce0f75-16d9-4f22-b475-2e5afaa9999a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:12.014469 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce0f75-16d9-4f22-b475-2e5afaa9999a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:12.014775 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce0f75-16d9-4f22-b475-2e5afaa9999a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:12.015056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce0f75-16d9-4f22-b475-2e5afaa9999a admin admin] PATH_INFO: `/v3/projects/c0af49a96daf40429b1952d158e36f6a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:12.016317 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce0f75-16d9-4f22-b475-2e5afaa9999a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0af49a96daf40429b1952d158e36f6a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:12.018315 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce0f75-16d9-4f22-b475-2e5afaa9999a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:12.066473 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cce0f75-16d9-4f22-b475-2e5afaa9999a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0af49a96daf40429b1952d158e36f6a', 'request_id': 'req-5cce0f75-16d9-4f22-b475-2e5afaa9999a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:12.072819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cce0f75-16d9-4f22-b475-2e5afaa9999a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0af49a96daf40429b1952d158e36f6a', 'request_id': 'req-5cce0f75-16d9-4f22-b475-2e5afaa9999a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:12.082468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cce0f75-16d9-4f22-b475-2e5afaa9999a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0af49a96daf40429b1952d158e36f6a', 'request_id': 'req-5cce0f75-16d9-4f22-b475-2e5afaa9999a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:12.110441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cce0f75-16d9-4f22-b475-2e5afaa9999a admin admin] The token cache is being invalidate because project c0af49a96daf40429b1952d158e36f6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:26:12.110932 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cce0f75-16d9-4f22-b475-2e5afaa9999a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:12.111703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cce0f75-16d9-4f22-b475-2e5afaa9999a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:12.114273 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1379/2749] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:26:11 2022] DELETE /identity/v3/projects/c0af49a96daf40429b1952d158e36f6a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:26:12.122184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9168e07-f8dd-4c94-b1bd-ac3f50f5aaa1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:12.122950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9168e07-f8dd-4c94-b1bd-ac3f50f5aaa1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:12.122950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9168e07-f8dd-4c94-b1bd-ac3f50f5aaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:12.141629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:26:12.149185 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c9168e07-f8dd-4c94-b1bd-ac3f50f5aaa1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:26:12.152100 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c9168e07-f8dd-4c94-b1bd-ac3f50f5aaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:12.213832 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1371/2750] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:26:12.222734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebfcac-9fa7-404e-a702-0bc37661465b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:12.223011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebfcac-9fa7-404e-a702-0bc37661465b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:12.223286 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebfcac-9fa7-404e-a702-0bc37661465b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:12.245489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-10ebfcac-9fa7-404e-a702-0bc37661465b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:26:12.246727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-10ebfcac-9fa7-404e-a702-0bc37661465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:12.255084 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1380/2751] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:26:12.259549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9168e07-f8dd-4c94-b1bd-ac3f50f5aaa1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:12.269937 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9168e07-f8dd-4c94-b1bd-ac3f50f5aaa1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:12.274478 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4294724-c97a-4507-af64-79b19c03d723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:12.275076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4294724-c97a-4507-af64-79b19c03d723 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:12.275352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4294724-c97a-4507-af64-79b19c03d723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:12.275607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4294724-c97a-4507-af64-79b19c03d723 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:12.276286 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4294724-c97a-4507-af64-79b19c03d723 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:26:12.276582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4294724-c97a-4507-af64-79b19c03d723 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:12.278221 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4294724-c97a-4507-af64-79b19c03d723 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:12.292731 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1372/2752] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:26:12.297207 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebfcac-9fa7-404e-a702-0bc37661465b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:12.306908 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebfcac-9fa7-404e-a702-0bc37661465b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:12.312240 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007c07f-3b30-4093-90b0-c0b0aeb4c968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:12.312886 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007c07f-3b30-4093-90b0-c0b0aeb4c968 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:12.313186 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007c07f-3b30-4093-90b0-c0b0aeb4c968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:12.313431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007c07f-3b30-4093-90b0-c0b0aeb4c968 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:12.314105 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007c07f-3b30-4093-90b0-c0b0aeb4c968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:12.316043 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007c07f-3b30-4093-90b0-c0b0aeb4c968 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:12.345293 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1381/2753] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:26:12 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:26:12.349488 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4294724-c97a-4507-af64-79b19c03d723 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:12.359699 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4294724-c97a-4507-af64-79b19c03d723 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:12.421528 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e753-a592-45ed-86fc-7d10e9297736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:12.422046 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e753-a592-45ed-86fc-7d10e9297736 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:12.422321 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e753-a592-45ed-86fc-7d10e9297736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:12.422596 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e753-a592-45ed-86fc-7d10e9297736 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:12.423220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434e753-a592-45ed-86fc-7d10e9297736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:12.425207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434e753-a592-45ed-86fc-7d10e9297736 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:12.457531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1373/2754] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:26:12 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:26:12.468231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007c07f-3b30-4093-90b0-c0b0aeb4c968 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:12.480041 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007c07f-3b30-4093-90b0-c0b0aeb4c968 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:12.484867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d0f2f-7da9-4239-92b3-e24e289382ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:12.485488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d0f2f-7da9-4239-92b3-e24e289382ef admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:12.485764 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d0f2f-7da9-4239-92b3-e24e289382ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:12.486027 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d0f2f-7da9-4239-92b3-e24e289382ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:12.486678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d0f2f-7da9-4239-92b3-e24e289382ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:12.488282 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d0f2f-7da9-4239-92b3-e24e289382ef admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:12.500931 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1382/2755] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:26:12.506163 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e753-a592-45ed-86fc-7d10e9297736 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:12.518524 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e753-a592-45ed-86fc-7d10e9297736 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:12.525322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e531a-5b61-4d37-82f8-ab6ac9dd057d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:12.526082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e531a-5b61-4d37-82f8-ab6ac9dd057d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:12.526496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e531a-5b61-4d37-82f8-ab6ac9dd057d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:12.526807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e531a-5b61-4d37-82f8-ab6ac9dd057d admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:12.527495 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e531a-5b61-4d37-82f8-ab6ac9dd057d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:12.529304 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e531a-5b61-4d37-82f8-ab6ac9dd057d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:12.541801 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1374/2756] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:26:12.547244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d0f2f-7da9-4239-92b3-e24e289382ef admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:12.557879 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d0f2f-7da9-4239-92b3-e24e289382ef admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:12.563219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd5048-b2fd-480c-9cfd-5a71a7494df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:12.563885 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd5048-b2fd-480c-9cfd-5a71a7494df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:12.564164 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd5048-b2fd-480c-9cfd-5a71a7494df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:12.564437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd5048-b2fd-480c-9cfd-5a71a7494df3 admin admin] PATH_INFO: `/v3/projects/ab5d76c603a04880887e1b85ba66e13b/users/e330014806dd4ff9ac98682c118de774/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:12.580393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd5048-b2fd-480c-9cfd-5a71a7494df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab5d76c603a04880887e1b85ba66e13b, user_id=e330014806dd4ff9ac98682c118de774, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:12.586347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd5048-b2fd-480c-9cfd-5a71a7494df3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:12.597946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1383/2757] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:26:12 2022] PUT /identity/v3/projects/ab5d76c603a04880887e1b85ba66e13b/users/e330014806dd4ff9ac98682c118de774/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:26:12.610635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859232f2-59f9-4147-b9cb-c7425083a0c7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:12.610915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859232f2-59f9-4147-b9cb-c7425083a0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:12.611218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859232f2-59f9-4147-b9cb-c7425083a0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:12.640633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-859232f2-59f9-4147-b9cb-c7425083a0c7 None None] MFA Rules not processed for user `e330014806dd4ff9ac98682c118de774`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:26:12.641922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-859232f2-59f9-4147-b9cb-c7425083a0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:12.725435 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1375/2758] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:26:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:26:12.734535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd5048-b2fd-480c-9cfd-5a71a7494df3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:12.736673 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8ccd5048-b2fd-480c-9cfd-5a71a7494df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:12.815714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:26:12.820341 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd5048-b2fd-480c-9cfd-5a71a7494df3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:12.824902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1364b9-cf0c-42ae-8fa1-fd00f989ae89 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a76937ab52e445c59b9eb9638fd4a753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:12.825552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1364b9-cf0c-42ae-8fa1-fd00f989ae89 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:12.825946 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1364b9-cf0c-42ae-8fa1-fd00f989ae89 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:12.827957 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1364b9-cf0c-42ae-8fa1-fd00f989ae89 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:12.837281 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1364b9-cf0c-42ae-8fa1-fd00f989ae89 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:12.839522 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1364b9-cf0c-42ae-8fa1-fd00f989ae89 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:12.852627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1384/2759] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:26:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:26:16.511121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859232f2-59f9-4147-b9cb-c7425083a0c7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:16.512952 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-859232f2-59f9-4147-b9cb-c7425083a0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:16.518819 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:26:16.633051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859232f2-59f9-4147-b9cb-c7425083a0c7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:16.639939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf213a-ab59-43c6-a375-b13991f275d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:16.640582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf213a-ab59-43c6-a375-b13991f275d3 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:16.641099 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf213a-ab59-43c6-a375-b13991f275d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:16.641381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf213a-ab59-43c6-a375-b13991f275d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:16.643264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0cbf213a-ab59-43c6-a375-b13991f275d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:16.673324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf213a-ab59-43c6-a375-b13991f275d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:16.675156 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf213a-ab59-43c6-a375-b13991f275d3 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:16.704964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1376/2760] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:26:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:26:22.847337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:26:26.696235 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=33 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:26:35.263297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9bf81-c02d-4ec3-8aab-07a8d0c94d8b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:35.263835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9bf81-c02d-4ec3-8aab-07a8d0c94d8b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:35.263835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9bf81-c02d-4ec3-8aab-07a8d0c94d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:35.285028 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-74d9bf81-c02d-4ec3-8aab-07a8d0c94d8b None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:26:35.286270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-74d9bf81-c02d-4ec3-8aab-07a8d0c94d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:35.350686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1385/2761] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:26:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:26:35.399467 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf213a-ab59-43c6-a375-b13991f275d3 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:35.400994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0cbf213a-ab59-43c6-a375-b13991f275d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:35.489849 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf213a-ab59-43c6-a375-b13991f275d3 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:35.495458 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5d9a1-abe2-4b2f-b1ae-6ae8cf14a6f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:35.496080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5d9a1-abe2-4b2f-b1ae-6ae8cf14a6f2 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:35.496359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5d9a1-abe2-4b2f-b1ae-6ae8cf14a6f2 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:35.496626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5d9a1-abe2-4b2f-b1ae-6ae8cf14a6f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:35.505025 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5d9a1-abe2-4b2f-b1ae-6ae8cf14a6f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:35.506604 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5d9a1-abe2-4b2f-b1ae-6ae8cf14a6f2 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:35.514777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1377/2762] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:26:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:26:44.189818 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6087c-c57d-4475-93e1-2256f8344afa None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:44.191048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6087c-c57d-4475-93e1-2256f8344afa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:44.191516 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6087c-c57d-4475-93e1-2256f8344afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:44.214221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1ac6087c-c57d-4475-93e1-2256f8344afa None None] MFA Rules not processed for user `e22472efcb9a454688744457cb45eabb`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:26:44.215579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1ac6087c-c57d-4475-93e1-2256f8344afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:44.283557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1386/2763] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:26:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:26:44.643310 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5d9a1-abe2-4b2f-b1ae-6ae8cf14a6f2 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:44.645045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7ce5d9a1-abe2-4b2f-b1ae-6ae8cf14a6f2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:44.720116 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5d9a1-abe2-4b2f-b1ae-6ae8cf14a6f2 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:44.724439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ee857-3265-4f84-9588-2cfbbf3baa7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:44.725227 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ee857-3265-4f84-9588-2cfbbf3baa7c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:44.725500 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ee857-3265-4f84-9588-2cfbbf3baa7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:44.725763 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ee857-3265-4f84-9588-2cfbbf3baa7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:44.727470 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-165ee857-3265-4f84-9588-2cfbbf3baa7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:44.741185 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ee857-3265-4f84-9588-2cfbbf3baa7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:44.742742 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ee857-3265-4f84-9588-2cfbbf3baa7c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:44.752849 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1378/2764] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:26:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:26:53.049693 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976e375-09ab-41f4-ae4c-56e9799b749a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:53.050371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976e375-09ab-41f4-ae4c-56e9799b749a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:53.050371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976e375-09ab-41f4-ae4c-56e9799b749a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:53.071331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3976e375-09ab-41f4-ae4c-56e9799b749a None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:26:53.072583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3976e375-09ab-41f4-ae4c-56e9799b749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:53.080284 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1387/2765] 10.222.0.79 () {58 vars in 967 bytes} [Tue Sep 6 21:26:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:26:53.092994 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ee857-3265-4f84-9588-2cfbbf3baa7c service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:53.105777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ee857-3265-4f84-9588-2cfbbf3baa7c service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:53.111161 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ed1e7-4922-45cc-baf1-4cf178f586d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:53.111712 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ed1e7-4922-45cc-baf1-4cf178f586d7 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:53.111967 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ed1e7-4922-45cc-baf1-4cf178f586d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:53.112246 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ed1e7-4922-45cc-baf1-4cf178f586d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:53.120635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ed1e7-4922-45cc-baf1-4cf178f586d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:53.122253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ed1e7-4922-45cc-baf1-4cf178f586d7 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:53.130398 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1379/2766] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:26:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:26:57.922925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976e375-09ab-41f4-ae4c-56e9799b749a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:57.938203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976e375-09ab-41f4-ae4c-56e9799b749a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:57.945303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f21f6e-668c-4806-80b2-df59627c9087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:57.945927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f21f6e-668c-4806-80b2-df59627c9087 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:57.946181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f21f6e-668c-4806-80b2-df59627c9087 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:57.946441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f21f6e-668c-4806-80b2-df59627c9087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:57.948470 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c0f21f6e-668c-4806-80b2-df59627c9087 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:57.961667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f21f6e-668c-4806-80b2-df59627c9087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:57.963270 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f21f6e-668c-4806-80b2-df59627c9087 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:57.974331 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1388/2767] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:26:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:26:59.405887 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ed1e7-4922-45cc-baf1-4cf178f586d7 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:59.407430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a35ed1e7-4922-45cc-baf1-4cf178f586d7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:26:59.413293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=70 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:26:59.484033 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ed1e7-4922-45cc-baf1-4cf178f586d7 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:59.488569 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a4bf7-2858-4a45-8af1-abd0d32ba6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:59.489258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a4bf7-2858-4a45-8af1-abd0d32ba6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:59.489536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a4bf7-2858-4a45-8af1-abd0d32ba6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:59.489806 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a4bf7-2858-4a45-8af1-abd0d32ba6cd admin admin] PATH_INFO: `/v3/users/525ba710ef8b40d0abb27bb2cad1095c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:59.490953 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a4bf7-2858-4a45-8af1-abd0d32ba6cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=525ba710ef8b40d0abb27bb2cad1095c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:59.492898 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a4bf7-2858-4a45-8af1-abd0d32ba6cd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:59.542093 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-349a4bf7-2858-4a45-8af1-abd0d32ba6cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '525ba710ef8b40d0abb27bb2cad1095c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:59.542655 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-349a4bf7-2858-4a45-8af1-abd0d32ba6cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '525ba710ef8b40d0abb27bb2cad1095c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:59.550442 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-349a4bf7-2858-4a45-8af1-abd0d32ba6cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '525ba710ef8b40d0abb27bb2cad1095c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:59.563881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-349a4bf7-2858-4a45-8af1-abd0d32ba6cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '525ba710ef8b40d0abb27bb2cad1095c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:59.564692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-349a4bf7-2858-4a45-8af1-abd0d32ba6cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '525ba710ef8b40d0abb27bb2cad1095c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:59.578848 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1380/2768] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:26:59 2022] DELETE /identity/v3/users/525ba710ef8b40d0abb27bb2cad1095c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:26:59.766418 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f21f6e-668c-4806-80b2-df59627c9087 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:26:59.783009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f21f6e-668c-4806-80b2-df59627c9087 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:26:59.847423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39453858-9a16-43e8-94b7-c6732319f925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:26:59.848085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39453858-9a16-43e8-94b7-c6732319f925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:59.848368 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39453858-9a16-43e8-94b7-c6732319f925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:59.848643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39453858-9a16-43e8-94b7-c6732319f925 admin admin] PATH_INFO: `/v3/projects/8c3a88bdee6e4566987b3e87b4627ca6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:59.849853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39453858-9a16-43e8-94b7-c6732319f925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c3a88bdee6e4566987b3e87b4627ca6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:26:59.851791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39453858-9a16-43e8-94b7-c6732319f925 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:26:59.889596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:26:59.902601 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-39453858-9a16-43e8-94b7-c6732319f925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c3a88bdee6e4566987b3e87b4627ca6', 'request_id': 'req-39453858-9a16-43e8-94b7-c6732319f925'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:59.909485 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-39453858-9a16-43e8-94b7-c6732319f925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c3a88bdee6e4566987b3e87b4627ca6', 'request_id': 'req-39453858-9a16-43e8-94b7-c6732319f925'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:59.916412 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-39453858-9a16-43e8-94b7-c6732319f925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c3a88bdee6e4566987b3e87b4627ca6', 'request_id': 'req-39453858-9a16-43e8-94b7-c6732319f925'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:59.929518 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-39453858-9a16-43e8-94b7-c6732319f925 admin admin] The token cache is being invalidate because project 8c3a88bdee6e4566987b3e87b4627ca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:26:59.929868 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-39453858-9a16-43e8-94b7-c6732319f925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:59.930705 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-39453858-9a16-43e8-94b7-c6732319f925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:26:59.932500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1389/2769] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:26:59 2022] DELETE /identity/v3/projects/8c3a88bdee6e4566987b3e87b4627ca6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:26:59.937902 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b1737-35d5-4d9b-868e-167e75e5d69b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:26:59.938192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b1737-35d5-4d9b-868e-167e75e5d69b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:26:59.938465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b1737-35d5-4d9b-868e-167e75e5d69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:26:59.958554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-199b1737-35d5-4d9b-868e-167e75e5d69b None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:26:59.959809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-199b1737-35d5-4d9b-868e-167e75e5d69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:00.025083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1381/2770] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:27:00.037223 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6269db0-ff35-434d-af9f-8e757970fdb6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:00.037606 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6269db0-ff35-434d-af9f-8e757970fdb6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:00.038057 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6269db0-ff35-434d-af9f-8e757970fdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:00.061960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c6269db0-ff35-434d-af9f-8e757970fdb6 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:27:00.063068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c6269db0-ff35-434d-af9f-8e757970fdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:00.071369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1390/2771] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:27:00.075906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b1737-35d5-4d9b-868e-167e75e5d69b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:00.093611 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b1737-35d5-4d9b-868e-167e75e5d69b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:00.099537 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8bf92-bede-47f6-86d1-b0f734ab0d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:00.100007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8bf92-bede-47f6-86d1-b0f734ab0d13 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:00.100283 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8bf92-bede-47f6-86d1-b0f734ab0d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:00.100554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8bf92-bede-47f6-86d1-b0f734ab0d13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:00.101250 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8bf92-bede-47f6-86d1-b0f734ab0d13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:27:00.101554 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8bf92-bede-47f6-86d1-b0f734ab0d13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:00.103371 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8bf92-bede-47f6-86d1-b0f734ab0d13 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:00.116320 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1382/2772] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:27:00.120961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6269db0-ff35-434d-af9f-8e757970fdb6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:00.131320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6269db0-ff35-434d-af9f-8e757970fdb6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:00.136930 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf509775-bdb5-4932-a633-ab0a143655f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:00.137549 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf509775-bdb5-4932-a633-ab0a143655f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:00.137824 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf509775-bdb5-4932-a633-ab0a143655f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:00.138090 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf509775-bdb5-4932-a633-ab0a143655f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:00.138725 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf509775-bdb5-4932-a633-ab0a143655f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:00.140821 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf509775-bdb5-4932-a633-ab0a143655f1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:00.183604 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1391/2773] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:27:00 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:27:00.188180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8bf92-bede-47f6-86d1-b0f734ab0d13 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:00.199439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8bf92-bede-47f6-86d1-b0f734ab0d13 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:00.205118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:27:00.270947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dd494-bb2a-4f4f-87bf-b86b230ef175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:00.271571 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dd494-bb2a-4f4f-87bf-b86b230ef175 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:00.271850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dd494-bb2a-4f4f-87bf-b86b230ef175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:00.272120 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dd494-bb2a-4f4f-87bf-b86b230ef175 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:00.272792 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83dd494-bb2a-4f4f-87bf-b86b230ef175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:00.275188 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83dd494-bb2a-4f4f-87bf-b86b230ef175 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:00.304753 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1383/2774] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:27:00 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:27:00.308969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf509775-bdb5-4932-a633-ab0a143655f1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:00.322156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf509775-bdb5-4932-a633-ab0a143655f1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:00.327531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1daf1-aeb3-4025-b4f2-84fbb4f17cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:00.328187 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1daf1-aeb3-4025-b4f2-84fbb4f17cab admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:00.328529 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1daf1-aeb3-4025-b4f2-84fbb4f17cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:00.328840 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1daf1-aeb3-4025-b4f2-84fbb4f17cab admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:00.329481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1daf1-aeb3-4025-b4f2-84fbb4f17cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:00.331309 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1daf1-aeb3-4025-b4f2-84fbb4f17cab admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:00.342802 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1392/2775] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:27:00.346985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dd494-bb2a-4f4f-87bf-b86b230ef175 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:00.362366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dd494-bb2a-4f4f-87bf-b86b230ef175 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:00.367512 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72406e73-fbd6-4861-a0bf-dd47de51c50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:00.368068 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72406e73-fbd6-4861-a0bf-dd47de51c50b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:00.368369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72406e73-fbd6-4861-a0bf-dd47de51c50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:00.368641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72406e73-fbd6-4861-a0bf-dd47de51c50b admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:00.369305 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72406e73-fbd6-4861-a0bf-dd47de51c50b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:00.370919 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72406e73-fbd6-4861-a0bf-dd47de51c50b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:00.382260 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1384/2776] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:27:00.386397 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1daf1-aeb3-4025-b4f2-84fbb4f17cab admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:00.397347 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1daf1-aeb3-4025-b4f2-84fbb4f17cab admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:00.402134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c1b26-a171-448a-84fd-262530da687e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:00.402811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c1b26-a171-448a-84fd-262530da687e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:00.403068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c1b26-a171-448a-84fd-262530da687e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:00.403351 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c1b26-a171-448a-84fd-262530da687e admin admin] PATH_INFO: `/v3/projects/ca88b1d59a924130a3fa43b0b256c33e/users/d52264275fdb411484cfc6b39b625d8b/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:00.434612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c1b26-a171-448a-84fd-262530da687e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca88b1d59a924130a3fa43b0b256c33e, user_id=d52264275fdb411484cfc6b39b625d8b, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:00.437435 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c1b26-a171-448a-84fd-262530da687e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:00.460730 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1393/2777] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:27:00 2022] PUT /identity/v3/projects/ca88b1d59a924130a3fa43b0b256c33e/users/d52264275fdb411484cfc6b39b625d8b/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:27:00.461923 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b0b4c-78c3-4af5-ac4e-ecf24bef0d4e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:00.462406 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b0b4c-78c3-4af5-ac4e-ecf24bef0d4e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:00.462516 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b0b4c-78c3-4af5-ac4e-ecf24bef0d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:00.482922 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-287b0b4c-78c3-4af5-ac4e-ecf24bef0d4e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:27:00.484169 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-287b0b4c-78c3-4af5-ac4e-ecf24bef0d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:00.545467 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1385/2778] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:27:00.555350 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c1b26-a171-448a-84fd-262530da687e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:00.557229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-231c1b26-a171-448a-84fd-262530da687e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:00.620344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c1b26-a171-448a-84fd-262530da687e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:00.624507 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db654922-f9e7-4692-b0c0-9d10c68cd269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:00.625133 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db654922-f9e7-4692-b0c0-9d10c68cd269 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:00.625417 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db654922-f9e7-4692-b0c0-9d10c68cd269 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:00.625687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db654922-f9e7-4692-b0c0-9d10c68cd269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:00.636207 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db654922-f9e7-4692-b0c0-9d10c68cd269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:00.637721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db654922-f9e7-4692-b0c0-9d10c68cd269 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:00.647592 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1394/2779] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:27:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:27:04.901383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039cc27-ae2e-4c8e-b38c-301965af4f98 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:04.901896 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039cc27-ae2e-4c8e-b38c-301965af4f98 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:04.901997 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039cc27-ae2e-4c8e-b38c-301965af4f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:04.951164 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6039cc27-ae2e-4c8e-b38c-301965af4f98 None None] MFA Rules not processed for user `d52264275fdb411484cfc6b39b625d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:27:04.952913 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6039cc27-ae2e-4c8e-b38c-301965af4f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:05.046547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1386/2780] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:27:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:27:05.051070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db654922-f9e7-4692-b0c0-9d10c68cd269 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:05.062762 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db654922-f9e7-4692-b0c0-9d10c68cd269 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:05.076036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd760e0-c270-4670-8962-322983854ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:05.076737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd760e0-c270-4670-8962-322983854ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:05.077022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd760e0-c270-4670-8962-322983854ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:05.077299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd760e0-c270-4670-8962-322983854ac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:05.078160 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd760e0-c270-4670-8962-322983854ac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:05.079936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd760e0-c270-4670-8962-322983854ac1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:05.115805 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1395/2781] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:27:05 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:27:05.120062 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039cc27-ae2e-4c8e-b38c-301965af4f98 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:05.132964 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039cc27-ae2e-4c8e-b38c-301965af4f98 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:05.150460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:27:05.208324 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd727810-bb15-4577-bd44-8ab7ad42593c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:05.208993 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd727810-bb15-4577-bd44-8ab7ad42593c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:05.209274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd727810-bb15-4577-bd44-8ab7ad42593c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:05.209539 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd727810-bb15-4577-bd44-8ab7ad42593c admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:05.210211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd727810-bb15-4577-bd44-8ab7ad42593c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:05.212191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd727810-bb15-4577-bd44-8ab7ad42593c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:05.240870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1387/2782] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:27:05 2022] POST /identity/v3/users => generated 299 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:27:05.243760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd760e0-c270-4670-8962-322983854ac1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:05.255944 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd760e0-c270-4670-8962-322983854ac1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:05.261015 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff9f93-952e-4e27-bf47-d4b45692b61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:05.261659 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff9f93-952e-4e27-bf47-d4b45692b61f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:05.261939 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff9f93-952e-4e27-bf47-d4b45692b61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:05.262212 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff9f93-952e-4e27-bf47-d4b45692b61f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:05.262879 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff9f93-952e-4e27-bf47-d4b45692b61f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:05.265895 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff9f93-952e-4e27-bf47-d4b45692b61f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:05.281857 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1396/2783] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:27:05.283058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd727810-bb15-4577-bd44-8ab7ad42593c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:05.294715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd727810-bb15-4577-bd44-8ab7ad42593c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:05.300600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867c26d-7882-4618-bcc5-253552746170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:05.301480 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867c26d-7882-4618-bcc5-253552746170 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:05.301745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867c26d-7882-4618-bcc5-253552746170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:05.302019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867c26d-7882-4618-bcc5-253552746170 admin admin] PATH_INFO: `/v3/projects/9ac7b59cce7e4f9fa9b3f04eb96acc9e/users/bdbc2fa907eb42509bf2517c648718bc/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:05.319921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867c26d-7882-4618-bcc5-253552746170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac7b59cce7e4f9fa9b3f04eb96acc9e, user_id=bdbc2fa907eb42509bf2517c648718bc, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:05.323119 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867c26d-7882-4618-bcc5-253552746170 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:05.338426 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1388/2784] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:27:05 2022] PUT /identity/v3/projects/9ac7b59cce7e4f9fa9b3f04eb96acc9e/users/bdbc2fa907eb42509bf2517c648718bc/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:27:05.342644 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff9f93-952e-4e27-bf47-d4b45692b61f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:05.354050 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff9f93-952e-4e27-bf47-d4b45692b61f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:05.393607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:27:05.445656 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70873428-0bf0-4fbd-a03a-189958fea918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:05.446300 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873428-0bf0-4fbd-a03a-189958fea918 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:05.446583 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873428-0bf0-4fbd-a03a-189958fea918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:05.446865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873428-0bf0-4fbd-a03a-189958fea918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:05.447571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70873428-0bf0-4fbd-a03a-189958fea918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:05.449430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70873428-0bf0-4fbd-a03a-189958fea918 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:05.464400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1397/2785] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:27:05.466225 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867c26d-7882-4618-bcc5-253552746170 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:05.479130 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867c26d-7882-4618-bcc5-253552746170 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:05.484135 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d1c41-70bf-4333-af33-cf6585ccac00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:05.484842 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d1c41-70bf-4333-af33-cf6585ccac00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:05.485117 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d1c41-70bf-4333-af33-cf6585ccac00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:05.485405 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d1c41-70bf-4333-af33-cf6585ccac00 admin admin] PATH_INFO: `/v3/projects/9ac7b59cce7e4f9fa9b3f04eb96acc9e/users/bdbc2fa907eb42509bf2517c648718bc/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:05.487340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d1c41-70bf-4333-af33-cf6585ccac00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac7b59cce7e4f9fa9b3f04eb96acc9e, user_id=bdbc2fa907eb42509bf2517c648718bc, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:05.490083 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d1c41-70bf-4333-af33-cf6585ccac00 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:05.501253 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1389/2786] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:27:05 2022] PUT /identity/v3/projects/9ac7b59cce7e4f9fa9b3f04eb96acc9e/users/bdbc2fa907eb42509bf2517c648718bc/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:27:09.886281 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941b6c-382a-4ebe-8666-c6fbdff878a6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:09.886865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941b6c-382a-4ebe-8666-c6fbdff878a6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:09.886865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941b6c-382a-4ebe-8666-c6fbdff878a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:09.919403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ba941b6c-382a-4ebe-8666-c6fbdff878a6 None None] MFA Rules not processed for user `bdbc2fa907eb42509bf2517c648718bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:27:09.920889 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ba941b6c-382a-4ebe-8666-c6fbdff878a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:09.992499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1398/2787] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:27:10.004928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d1c41-70bf-4333-af33-cf6585ccac00 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:10.007154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-617d1c41-70bf-4333-af33-cf6585ccac00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:10.073944 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d1c41-70bf-4333-af33-cf6585ccac00 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:10.078201 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70bf05-6b40-420b-bb88-53a91b03921e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a76937ab52e445c59b9eb9638fd4a753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:10.078820 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70bf05-6b40-420b-bb88-53a91b03921e service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:10.079104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70bf05-6b40-420b-bb88-53a91b03921e service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:10.079385 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70bf05-6b40-420b-bb88-53a91b03921e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:10.139284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70bf05-6b40-420b-bb88-53a91b03921e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:10.140775 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70bf05-6b40-420b-bb88-53a91b03921e service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:10.150431 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1390/2788] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:27:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:27:10.292099 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597dfb0-353f-4c92-a83d-2d10f3c7a9db None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:10.292388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597dfb0-353f-4c92-a83d-2d10f3c7a9db None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:10.292687 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597dfb0-353f-4c92-a83d-2d10f3c7a9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:10.312876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3597dfb0-353f-4c92-a83d-2d10f3c7a9db None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:27:10.314263 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3597dfb0-353f-4c92-a83d-2d10f3c7a9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:10.375535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1399/2789] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:27:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:27:10.398912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70bf05-6b40-420b-bb88-53a91b03921e service glance] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:10.398912 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2b70bf05-6b40-420b-bb88-53a91b03921e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:10.442502 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:27:10.503568 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70bf05-6b40-420b-bb88-53a91b03921e service glance] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:10.508935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db95d5-997b-4c8b-964c-4bbad0d0313d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:10.509629 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db95d5-997b-4c8b-964c-4bbad0d0313d service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:10.509904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db95d5-997b-4c8b-964c-4bbad0d0313d service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:10.510216 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db95d5-997b-4c8b-964c-4bbad0d0313d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:10.526839 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db95d5-997b-4c8b-964c-4bbad0d0313d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:10.528481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db95d5-997b-4c8b-964c-4bbad0d0313d service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:10.542584 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1391/2790] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:27:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:27:12.788638 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597dfb0-353f-4c92-a83d-2d10f3c7a9db None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:12.792593 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3597dfb0-353f-4c92-a83d-2d10f3c7a9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:12.868980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:27:12.910822 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597dfb0-353f-4c92-a83d-2d10f3c7a9db None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:12.925210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:12.926233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:12.926517 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:12.927976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] PATH_INFO: `/v3/users/e330014806dd4ff9ac98682c118de774` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:12.931679 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e330014806dd4ff9ac98682c118de774)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:12.937828 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:13.059871 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e330014806dd4ff9ac98682c118de774'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:13.062846 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e330014806dd4ff9ac98682c118de774'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:13.071468 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e330014806dd4ff9ac98682c118de774'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:13.096068 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e330014806dd4ff9ac98682c118de774'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:13.099913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e330014806dd4ff9ac98682c118de774'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:13.127048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1400/2791] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:27:12 2022] DELETE /identity/v3/users/e330014806dd4ff9ac98682c118de774 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:27:13.131356 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38134b7-a4aa-4550-aca8-7cbb9bb1b76f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:13.137612 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38134b7-a4aa-4550-aca8-7cbb9bb1b76f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:13.138178 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38134b7-a4aa-4550-aca8-7cbb9bb1b76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:13.200210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e38134b7-a4aa-4550-aca8-7cbb9bb1b76f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:27:13.204317 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e38134b7-a4aa-4550-aca8-7cbb9bb1b76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:13.343610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1392/2792] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:27:13.351642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:13.353278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:13.456405 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57902c4-be86-41a9-bc04-669d8624d09a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:13.466028 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d95e9-88c5-4bc4-ab18-5684d02ea100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:13.466976 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d95e9-88c5-4bc4-ab18-5684d02ea100 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:13.467385 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d95e9-88c5-4bc4-ab18-5684d02ea100 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:13.467779 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d95e9-88c5-4bc4-ab18-5684d02ea100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:13.482622 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d95e9-88c5-4bc4-ab18-5684d02ea100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:13.485225 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d95e9-88c5-4bc4-ab18-5684d02ea100 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:13.502960 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1401/2793] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:27:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:27:13.856430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38134b7-a4aa-4550-aca8-7cbb9bb1b76f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:13.858361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e38134b7-a4aa-4550-aca8-7cbb9bb1b76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:13.874879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38134b7-a4aa-4550-aca8-7cbb9bb1b76f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:13.881545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:13.882208 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:13.882482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:13.882749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a admin admin] PATH_INFO: `/v3/projects/ab5d76c603a04880887e1b85ba66e13b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:13.884162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab5d76c603a04880887e1b85ba66e13b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:13.886722 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:13.984604 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab5d76c603a04880887e1b85ba66e13b', 'request_id': 'req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:13.995973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab5d76c603a04880887e1b85ba66e13b', 'request_id': 'req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:14.004719 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab5d76c603a04880887e1b85ba66e13b', 'request_id': 'req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:14.017650 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a admin admin] The token cache is being invalidate because project ab5d76c603a04880887e1b85ba66e13b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:27:14.020412 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:14.020804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c7e5ce77-2dfe-4d28-8bc8-35a1dc334a4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:14.022929 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1393/2794] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:27:13 2022] DELETE /identity/v3/projects/ab5d76c603a04880887e1b85ba66e13b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:27:19.078496 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d95e9-88c5-4bc4-ab18-5684d02ea100 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:19.080058 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a33d95e9-88c5-4bc4-ab18-5684d02ea100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:19.171108 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d95e9-88c5-4bc4-ab18-5684d02ea100 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:19.185372 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:27:19.242501 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d8f3d-00f8-49ec-a8e7-59c419c4a15d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:19.243124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d8f3d-00f8-49ec-a8e7-59c419c4a15d service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:19.243481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d8f3d-00f8-49ec-a8e7-59c419c4a15d service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:19.243751 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d8f3d-00f8-49ec-a8e7-59c419c4a15d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:19.245619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7d3d8f3d-00f8-49ec-a8e7-59c419c4a15d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:19.257913 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d8f3d-00f8-49ec-a8e7-59c419c4a15d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:19.259522 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d8f3d-00f8-49ec-a8e7-59c419c4a15d service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:19.271884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1402/2795] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:27:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:27:20.396032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d434cd-c8e6-4673-a2a6-ee324105443e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:20.396590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d434cd-c8e6-4673-a2a6-ee324105443e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:20.396590 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d434cd-c8e6-4673-a2a6-ee324105443e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:20.416641 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c8d434cd-c8e6-4673-a2a6-ee324105443e None None] MFA Rules not processed for user `d52264275fdb411484cfc6b39b625d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:27:20.417901 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c8d434cd-c8e6-4673-a2a6-ee324105443e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:20.456627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:27:20.485890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1394/2796] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:27:22.897748 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858806e-4849-4bc6-b37b-3fd7f63f332a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:22.898834 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858806e-4849-4bc6-b37b-3fd7f63f332a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:22.899134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858806e-4849-4bc6-b37b-3fd7f63f332a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:22.920147 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1858806e-4849-4bc6-b37b-3fd7f63f332a None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:27:22.921388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1858806e-4849-4bc6-b37b-3fd7f63f332a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:22.985702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1403/2797] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Sep 6 21:27:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:27:23.000547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d434cd-c8e6-4673-a2a6-ee324105443e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:23.002720 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c8d434cd-c8e6-4673-a2a6-ee324105443e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:23.078777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d434cd-c8e6-4673-a2a6-ee324105443e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:23.084167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c0909-8860-4a0d-889e-5aba36715010 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:23.084824 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c0909-8860-4a0d-889e-5aba36715010 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:23.085106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c0909-8860-4a0d-889e-5aba36715010 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:23.085380 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c0909-8860-4a0d-889e-5aba36715010 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:23.095798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c0909-8860-4a0d-889e-5aba36715010 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:23.097407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c0909-8860-4a0d-889e-5aba36715010 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:23.106352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1395/2798] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:27:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:27:32.977938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=49 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:27:33.102626 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=40 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:27:44.646461 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858806e-4849-4bc6-b37b-3fd7f63f332a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:44.648208 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1858806e-4849-4bc6-b37b-3fd7f63f332a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:44.710984 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858806e-4849-4bc6-b37b-3fd7f63f332a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:44.714712 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd8a11-db4e-442d-b99b-d2f32caa3d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:44.715319 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd8a11-db4e-442d-b99b-d2f32caa3d12 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:44.715595 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd8a11-db4e-442d-b99b-d2f32caa3d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:44.715843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd8a11-db4e-442d-b99b-d2f32caa3d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:44.717927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4dbd8a11-db4e-442d-b99b-d2f32caa3d12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:44.780437 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd8a11-db4e-442d-b99b-d2f32caa3d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:44.781932 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd8a11-db4e-442d-b99b-d2f32caa3d12 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:44.792045 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1404/2799] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:27:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:27:51.534853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c0909-8860-4a0d-889e-5aba36715010 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:51.536466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-176c0909-8860-4a0d-889e-5aba36715010 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:51.551870 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c0909-8860-4a0d-889e-5aba36715010 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:51.558018 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:51.558668 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:51.558948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:51.559221 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] PATH_INFO: `/v3/users/6d0c1c510b0a4fa39927f7f92352d034` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:51.573476 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d0c1c510b0a4fa39927f7f92352d034)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:51.575429 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:51.625366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d0c1c510b0a4fa39927f7f92352d034'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:51.626013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d0c1c510b0a4fa39927f7f92352d034'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:51.633043 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d0c1c510b0a4fa39927f7f92352d034'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:51.644939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d0c1c510b0a4fa39927f7f92352d034'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:51.645593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d0c1c510b0a4fa39927f7f92352d034'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:51.659807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1396/2800] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:27:51 2022] DELETE /identity/v3/users/6d0c1c510b0a4fa39927f7f92352d034 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:27:51.843758 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd8a11-db4e-442d-b99b-d2f32caa3d12 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:51.854411 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd8a11-db4e-442d-b99b-d2f32caa3d12 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:51.912798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:51.913458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:51.913750 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:51.914018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] PATH_INFO: `/v3/projects/5801984f41a043daa52dbef9de89df9a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:51.926459 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5801984f41a043daa52dbef9de89df9a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:51.928489 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:51.978219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:27:51.982243 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5801984f41a043daa52dbef9de89df9a', 'request_id': 'req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:51.990278 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5801984f41a043daa52dbef9de89df9a', 'request_id': 'req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:51.998004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5801984f41a043daa52dbef9de89df9a', 'request_id': 'req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:52.013618 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] The token cache is being invalidate because project 5801984f41a043daa52dbef9de89df9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:27:52.013905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:52.014525 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:52.016024 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1405/2801] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:27:51 2022] DELETE /identity/v3/projects/5801984f41a043daa52dbef9de89df9a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:27:52.020330 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:52.021904 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:52.036117 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdce3fe-a0b6-449d-b95c-50e9dcfc9f94 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:52.097294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:52.097918 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:52.098192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:52.098469 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] PATH_INFO: `/v3/users/d782519572b34fadbbab114d96152eda` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:52.112985 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d782519572b34fadbbab114d96152eda)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:52.114919 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:52.164642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd782519572b34fadbbab114d96152eda'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:52.165698 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd782519572b34fadbbab114d96152eda'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:52.170132 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:27:52.173223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd782519572b34fadbbab114d96152eda'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:52.186410 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd782519572b34fadbbab114d96152eda'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:52.187047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd782519572b34fadbbab114d96152eda'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:52.200614 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1397/2802] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:27:52 2022] DELETE /identity/v3/users/d782519572b34fadbbab114d96152eda => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:27:52.395070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:52.396728 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:52.412130 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce9a38-e7bb-4312-9dc3-e4b2336c5a89 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:52.471382 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:52.472067 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:52.472371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:52.472647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] PATH_INFO: `/v3/projects/a6e7246e3e3247bfa91d9122dbb59226` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:52.486615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e7246e3e3247bfa91d9122dbb59226)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:52.488594 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:52.551351 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e7246e3e3247bfa91d9122dbb59226', 'request_id': 'req-6194f1cd-add6-4ef0-a12f-4ab17c03532c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:52.559532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e7246e3e3247bfa91d9122dbb59226', 'request_id': 'req-6194f1cd-add6-4ef0-a12f-4ab17c03532c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:52.565936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e7246e3e3247bfa91d9122dbb59226', 'request_id': 'req-6194f1cd-add6-4ef0-a12f-4ab17c03532c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:52.582311 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] The token cache is being invalidate because project a6e7246e3e3247bfa91d9122dbb59226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:27:52.583297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:52.584029 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:27:52.585809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1406/2803] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:27:52 2022] DELETE /identity/v3/projects/a6e7246e3e3247bfa91d9122dbb59226 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:27:57.190649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:27:57.192366 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:57.252683 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bb3d1-a558-437c-8391-04f1f293b81f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:27:57.306383 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89792a-4211-4a83-8557-fefed47da807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:27:57.306978 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89792a-4211-4a83-8557-fefed47da807 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:27:57.307248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89792a-4211-4a83-8557-fefed47da807 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:27:57.307514 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89792a-4211-4a83-8557-fefed47da807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:27:57.309726 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7c89792a-4211-4a83-8557-fefed47da807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:27:57.383155 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89792a-4211-4a83-8557-fefed47da807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:27:57.384525 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89792a-4211-4a83-8557-fefed47da807 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:27:57.395001 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1398/2804] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:27:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:28:02.578823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=63,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:28:07.390446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=80,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:28:17.965560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:28:17.967137 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:28:18.045572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194f1cd-add6-4ef0-a12f-4ab17c03532c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:28:18.051647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daa3d3-388b-438d-b3ae-2e65144ec2d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12d51454d20c46d7a5e6d963a2f987cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:28:18.052258 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daa3d3-388b-438d-b3ae-2e65144ec2d4 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:28:18.052536 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daa3d3-388b-438d-b3ae-2e65144ec2d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:28:18.052840 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daa3d3-388b-438d-b3ae-2e65144ec2d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:28:18.055014 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-68daa3d3-388b-438d-b3ae-2e65144ec2d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:28:18.112585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68daa3d3-388b-438d-b3ae-2e65144ec2d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:28:18.114494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68daa3d3-388b-438d-b3ae-2e65144ec2d4 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:28:18.123596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1407/2805] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:28:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:28:28.119777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=56 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:28:35.834748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fe8bf-697f-4467-8602-d8a39f3ec0d6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:28:35.835344 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fe8bf-697f-4467-8602-d8a39f3ec0d6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:28:35.835344 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fe8bf-697f-4467-8602-d8a39f3ec0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:28:35.857082 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e40fe8bf-697f-4467-8602-d8a39f3ec0d6 None None] MFA Rules not processed for user `e22472efcb9a454688744457cb45eabb`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:28:35.858334 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e40fe8bf-697f-4467-8602-d8a39f3ec0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:28:35.866714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1399/2806] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:28:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:28:43.913581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daa3d3-388b-438d-b3ae-2e65144ec2d4 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:28:43.915520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-68daa3d3-388b-438d-b3ae-2e65144ec2d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:28:43.928359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daa3d3-388b-438d-b3ae-2e65144ec2d4 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:28:43.932419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750a1c3-ecc0-4b0a-80be-232b407703ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:28:43.933126 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750a1c3-ecc0-4b0a-80be-232b407703ee service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:28:43.933400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750a1c3-ecc0-4b0a-80be-232b407703ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:28:43.933672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750a1c3-ecc0-4b0a-80be-232b407703ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:28:43.935981 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4750a1c3-ecc0-4b0a-80be-232b407703ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:28:43.952852 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750a1c3-ecc0-4b0a-80be-232b407703ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:28:43.955236 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750a1c3-ecc0-4b0a-80be-232b407703ee service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:28:43.970166 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1408/2807] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:28:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:28:45.174259 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fe8bf-697f-4467-8602-d8a39f3ec0d6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:28:45.176007 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e40fe8bf-697f-4467-8602-d8a39f3ec0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:28:45.189652 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fe8bf-697f-4467-8602-d8a39f3ec0d6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:28:45.195946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c80a4-e6a4-42fd-ad94-9b6d01320693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:28:45.196932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c80a4-e6a4-42fd-ad94-9b6d01320693 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:28:45.197247 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c80a4-e6a4-42fd-ad94-9b6d01320693 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:28:45.197589 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c80a4-e6a4-42fd-ad94-9b6d01320693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:28:45.199787 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e49c80a4-e6a4-42fd-ad94-9b6d01320693 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:28:45.214465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c80a4-e6a4-42fd-ad94-9b6d01320693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:28:45.215905 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c80a4-e6a4-42fd-ad94-9b6d01320693 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:28:45.227703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1400/2808] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:28:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:28:53.964924 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:28:55.222790 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=9 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:03.363409 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750a1c3-ecc0-4b0a-80be-232b407703ee service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:03.372896 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750a1c3-ecc0-4b0a-80be-232b407703ee service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:03.377560 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c77f7-3c15-4472-a166-2ff01c6b84a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:03.378221 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c77f7-3c15-4472-a166-2ff01c6b84a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:03.378499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c77f7-3c15-4472-a166-2ff01c6b84a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:03.378770 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c77f7-3c15-4472-a166-2ff01c6b84a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:03.380884 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1e4c77f7-3c15-4472-a166-2ff01c6b84a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:03.391686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c77f7-3c15-4472-a166-2ff01c6b84a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:03.393196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c77f7-3c15-4472-a166-2ff01c6b84a4 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:03.401518 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1409/2809] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:30:13.397804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:25.708900 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c80a4-e6a4-42fd-ad94-9b6d01320693 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:25.719628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c80a4-e6a4-42fd-ad94-9b6d01320693 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:25.723873 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e19e5d-fc80-4577-9b77-857e57162c12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:25.724610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e19e5d-fc80-4577-9b77-857e57162c12 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:25.727315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e19e5d-fc80-4577-9b77-857e57162c12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:25.727315 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e19e5d-fc80-4577-9b77-857e57162c12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:25.729256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-20e19e5d-fc80-4577-9b77-857e57162c12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:25.806167 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e19e5d-fc80-4577-9b77-857e57162c12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:25.807814 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e19e5d-fc80-4577-9b77-857e57162c12 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:25.817975 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1401/2810] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:30:26.876481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3d501-c294-49dc-b3f6-374f544b8c6a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:26.877119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3d501-c294-49dc-b3f6-374f544b8c6a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:26.877478 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3d501-c294-49dc-b3f6-374f544b8c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:26.900500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-00b3d501-c294-49dc-b3f6-374f544b8c6a None None] MFA Rules not processed for user `c09b4900d97b4b35b617af9a42baf9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:26.901704 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-00b3d501-c294-49dc-b3f6-374f544b8c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:26.988104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1410/2811] 10.222.0.79 () {58 vars in 967 bytes} [Tue Sep 6 21:30:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:27.064123 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e19e5d-fc80-4577-9b77-857e57162c12 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:27.066745 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-20e19e5d-fc80-4577-9b77-857e57162c12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:27.166778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e19e5d-fc80-4577-9b77-857e57162c12 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:27.172110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b873a3-c72f-4588-9be7-6c139a3e2093 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061c584de8264868af8782402a98a72e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:27.172760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b873a3-c72f-4588-9be7-6c139a3e2093 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:27.173037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b873a3-c72f-4588-9be7-6c139a3e2093 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:27.173293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b873a3-c72f-4588-9be7-6c139a3e2093 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:27.184274 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b873a3-c72f-4588-9be7-6c139a3e2093 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:27.185774 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b873a3-c72f-4588-9be7-6c139a3e2093 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:27.196045 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1402/2812] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Sep 6 21:30:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 21:30:29.403636 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3d501-c294-49dc-b3f6-374f544b8c6a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:29.407219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-00b3d501-c294-49dc-b3f6-374f544b8c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:29.438241 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3d501-c294-49dc-b3f6-374f544b8c6a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:29.448926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5ee7c-8d1d-4762-8e56-6ba05c30eacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:29.449788 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5ee7c-8d1d-4762-8e56-6ba05c30eacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:29.450837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5ee7c-8d1d-4762-8e56-6ba05c30eacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:29.452611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5ee7c-8d1d-4762-8e56-6ba05c30eacb admin admin] PATH_INFO: `/v3/users/e22472efcb9a454688744457cb45eabb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:29.456854 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5ee7c-8d1d-4762-8e56-6ba05c30eacb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e22472efcb9a454688744457cb45eabb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:29.465020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5ee7c-8d1d-4762-8e56-6ba05c30eacb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:29.528928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3d5ee7c-8d1d-4762-8e56-6ba05c30eacb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e22472efcb9a454688744457cb45eabb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:29.530030 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3d5ee7c-8d1d-4762-8e56-6ba05c30eacb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e22472efcb9a454688744457cb45eabb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:29.544351 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3d5ee7c-8d1d-4762-8e56-6ba05c30eacb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e22472efcb9a454688744457cb45eabb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:29.566612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3d5ee7c-8d1d-4762-8e56-6ba05c30eacb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e22472efcb9a454688744457cb45eabb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:29.567692 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3d5ee7c-8d1d-4762-8e56-6ba05c30eacb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e22472efcb9a454688744457cb45eabb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:29.608843 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1411/2813] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:30:29 2022] DELETE /identity/v3/users/e22472efcb9a454688744457cb45eabb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:29.847700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b873a3-c72f-4588-9be7-6c139a3e2093 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:29.867195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b873a3-c72f-4588-9be7-6c139a3e2093 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:29.951567 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbca7c-7d06-4242-821a-e724bee73566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:29.952413 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbca7c-7d06-4242-821a-e724bee73566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:29.952822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbca7c-7d06-4242-821a-e724bee73566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:29.953207 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbca7c-7d06-4242-821a-e724bee73566 admin admin] PATH_INFO: `/v3/projects/0ca0b63bd9224370b45e7cc5e9c68a27` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:29.954465 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dbca7c-7d06-4242-821a-e724bee73566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ca0b63bd9224370b45e7cc5e9c68a27)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:29.956707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dbca7c-7d06-4242-821a-e724bee73566 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:30.012208 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:30.051341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3dbca7c-7d06-4242-821a-e724bee73566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ca0b63bd9224370b45e7cc5e9c68a27', 'request_id': 'req-c3dbca7c-7d06-4242-821a-e724bee73566'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:30.066840 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3dbca7c-7d06-4242-821a-e724bee73566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ca0b63bd9224370b45e7cc5e9c68a27', 'request_id': 'req-c3dbca7c-7d06-4242-821a-e724bee73566'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:30.077037 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3dbca7c-7d06-4242-821a-e724bee73566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ca0b63bd9224370b45e7cc5e9c68a27', 'request_id': 'req-c3dbca7c-7d06-4242-821a-e724bee73566'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:30.095818 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3dbca7c-7d06-4242-821a-e724bee73566 admin admin] The token cache is being invalidate because project 0ca0b63bd9224370b45e7cc5e9c68a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:30:30.096345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3dbca7c-7d06-4242-821a-e724bee73566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:30.097331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3dbca7c-7d06-4242-821a-e724bee73566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:30.099409 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1403/2814] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:30:29 2022] DELETE /identity/v3/projects/0ca0b63bd9224370b45e7cc5e9c68a27 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:30.104611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82f8ff-f560-4491-98f4-c3e1ccf1459c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:30.105147 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82f8ff-f560-4491-98f4-c3e1ccf1459c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:30.105589 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82f8ff-f560-4491-98f4-c3e1ccf1459c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:30.143683 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5c82f8ff-f560-4491-98f4-c3e1ccf1459c None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:30.145619 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5c82f8ff-f560-4491-98f4-c3e1ccf1459c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:30.224633 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1412/2815] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:30:30.233296 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b66de8-ce1e-4e60-bea1-fcc55a3edbd5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:30.233653 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b66de8-ce1e-4e60-bea1-fcc55a3edbd5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:30.234006 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b66de8-ce1e-4e60-bea1-fcc55a3edbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:30.256362 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e1b66de8-ce1e-4e60-bea1-fcc55a3edbd5 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:30.258705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e1b66de8-ce1e-4e60-bea1-fcc55a3edbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:30.274294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1404/2816] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:30:30.278955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82f8ff-f560-4491-98f4-c3e1ccf1459c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:30.290968 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82f8ff-f560-4491-98f4-c3e1ccf1459c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:30.298770 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e825887-f450-404e-a7a4-6413322cb9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:30.301303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e825887-f450-404e-a7a4-6413322cb9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:30.301597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e825887-f450-404e-a7a4-6413322cb9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:30.301886 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e825887-f450-404e-a7a4-6413322cb9e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:30.302555 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e825887-f450-404e-a7a4-6413322cb9e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:30:30.302863 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e825887-f450-404e-a7a4-6413322cb9e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:30.304762 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e825887-f450-404e-a7a4-6413322cb9e5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:30.319435 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1413/2817] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:30:30.322417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b66de8-ce1e-4e60-bea1-fcc55a3edbd5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:30.334278 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b66de8-ce1e-4e60-bea1-fcc55a3edbd5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:30.340282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8e9a2-62f4-42be-b51f-5c64e6bbf346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:30.341010 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8e9a2-62f4-42be-b51f-5c64e6bbf346 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:30.341293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8e9a2-62f4-42be-b51f-5c64e6bbf346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:30.341588 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8e9a2-62f4-42be-b51f-5c64e6bbf346 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:30.342410 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8e9a2-62f4-42be-b51f-5c64e6bbf346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:30.344430 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8e9a2-62f4-42be-b51f-5c64e6bbf346 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:30.380163 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1405/2818] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:30:30 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:30.385196 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e825887-f450-404e-a7a4-6413322cb9e5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:30.393440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:30.397782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e825887-f450-404e-a7a4-6413322cb9e5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:30.458791 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf00bc1-b05a-4be4-a347-c11fa5e0971a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:30.459521 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf00bc1-b05a-4be4-a347-c11fa5e0971a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:30.459726 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf00bc1-b05a-4be4-a347-c11fa5e0971a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:30.459983 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf00bc1-b05a-4be4-a347-c11fa5e0971a admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:30.461117 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf00bc1-b05a-4be4-a347-c11fa5e0971a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:30.462669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf00bc1-b05a-4be4-a347-c11fa5e0971a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:30.487480 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1414/2819] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:30:30 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:30.492079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8e9a2-62f4-42be-b51f-5c64e6bbf346 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:30.506270 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8e9a2-62f4-42be-b51f-5c64e6bbf346 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:30.512140 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ded1f-ba27-4198-87ef-c8d8ba8ca827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:30.512834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ded1f-ba27-4198-87ef-c8d8ba8ca827 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:30.513105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ded1f-ba27-4198-87ef-c8d8ba8ca827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:30.513377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ded1f-ba27-4198-87ef-c8d8ba8ca827 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:30.514029 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ded1f-ba27-4198-87ef-c8d8ba8ca827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:30.515665 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ded1f-ba27-4198-87ef-c8d8ba8ca827 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:30.527288 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1406/2820] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:30.531286 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf00bc1-b05a-4be4-a347-c11fa5e0971a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:30.542325 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf00bc1-b05a-4be4-a347-c11fa5e0971a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:30.548190 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3d77e-0499-4087-8941-444884d7cbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:30.548856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d77e-0499-4087-8941-444884d7cbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:30.549131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d77e-0499-4087-8941-444884d7cbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:30.549394 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d77e-0499-4087-8941-444884d7cbd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:30.550053 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3d77e-0499-4087-8941-444884d7cbd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:30.551678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3d77e-0499-4087-8941-444884d7cbd2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:30.566333 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1415/2821] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:30.569697 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ded1f-ba27-4198-87ef-c8d8ba8ca827 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:30.583285 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ded1f-ba27-4198-87ef-c8d8ba8ca827 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:30.593981 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b5501-40f0-4139-88bd-8a0abe5297f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:30.594675 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b5501-40f0-4139-88bd-8a0abe5297f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:30.594959 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b5501-40f0-4139-88bd-8a0abe5297f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:30.595229 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b5501-40f0-4139-88bd-8a0abe5297f9 admin admin] PATH_INFO: `/v3/projects/a2104705c6514c9e94c10fedf14f31ee/users/9901966e03ae4edda0042fc11d09d545/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:30.618258 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b5501-40f0-4139-88bd-8a0abe5297f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2104705c6514c9e94c10fedf14f31ee, user_id=9901966e03ae4edda0042fc11d09d545, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:30.621197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b5501-40f0-4139-88bd-8a0abe5297f9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:30.648069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1407/2822] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:30 2022] PUT /identity/v3/projects/a2104705c6514c9e94c10fedf14f31ee/users/9901966e03ae4edda0042fc11d09d545/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:30:30.658523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218662ba-3bfc-4484-bbf5-8fbe409caed5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:30.658936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218662ba-3bfc-4484-bbf5-8fbe409caed5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:30.659224 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218662ba-3bfc-4484-bbf5-8fbe409caed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:30.699174 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-218662ba-3bfc-4484-bbf5-8fbe409caed5 None None] MFA Rules not processed for user `9901966e03ae4edda0042fc11d09d545`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:30.700602 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-218662ba-3bfc-4484-bbf5-8fbe409caed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:30.803552 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1416/2823] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:30.808439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b5501-40f0-4139-88bd-8a0abe5297f9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:30.887136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b5501-40f0-4139-88bd-8a0abe5297f9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:30.892181 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe884a98-3228-499c-b47c-36e118d31656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:30.892915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe884a98-3228-499c-b47c-36e118d31656 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:30.893286 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe884a98-3228-499c-b47c-36e118d31656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:30.893650 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe884a98-3228-499c-b47c-36e118d31656 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:30.894402 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe884a98-3228-499c-b47c-36e118d31656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:30.896948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe884a98-3228-499c-b47c-36e118d31656 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:30.931311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1408/2824] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:30:30 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:30.935603 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218662ba-3bfc-4484-bbf5-8fbe409caed5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:30.947393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218662ba-3bfc-4484-bbf5-8fbe409caed5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:31.047109 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f3511-0b5e-42b1-8969-121eb2c923f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:31.047772 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f3511-0b5e-42b1-8969-121eb2c923f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:31.048048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f3511-0b5e-42b1-8969-121eb2c923f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:31.048332 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f3511-0b5e-42b1-8969-121eb2c923f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:31.049686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f3511-0b5e-42b1-8969-121eb2c923f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:31.052214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f3511-0b5e-42b1-8969-121eb2c923f7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:31.081419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1417/2825] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:30:30 2022] POST /identity/v3/users => generated 290 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:31.085635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe884a98-3228-499c-b47c-36e118d31656 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:31.098914 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe884a98-3228-499c-b47c-36e118d31656 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:31.109054 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ea060-41e2-43de-83fd-510d4ff8e11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:31.118856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ea060-41e2-43de-83fd-510d4ff8e11b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:31.119804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ea060-41e2-43de-83fd-510d4ff8e11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:31.120277 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ea060-41e2-43de-83fd-510d4ff8e11b admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:31.122180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ea060-41e2-43de-83fd-510d4ff8e11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:31.124151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ea060-41e2-43de-83fd-510d4ff8e11b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:31.148727 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1409/2826] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:31.148876 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f3511-0b5e-42b1-8969-121eb2c923f7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:31.155413 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:31.161349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f3511-0b5e-42b1-8969-121eb2c923f7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:31.171442 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dc357-e70d-43d7-89cf-d823f7b2cfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:31.172283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dc357-e70d-43d7-89cf-d823f7b2cfa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:31.172736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dc357-e70d-43d7-89cf-d823f7b2cfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:31.173168 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dc357-e70d-43d7-89cf-d823f7b2cfa5 admin admin] PATH_INFO: `/v3/projects/ae1cf229e3bf465cb5e2ec9c6251ece5/users/388a1e08eb1a4aa7b41756b0664a0725/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:31.191660 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913dc357-e70d-43d7-89cf-d823f7b2cfa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae1cf229e3bf465cb5e2ec9c6251ece5, user_id=388a1e08eb1a4aa7b41756b0664a0725, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:31.194427 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913dc357-e70d-43d7-89cf-d823f7b2cfa5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:31.216690 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1418/2827] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:31 2022] PUT /identity/v3/projects/ae1cf229e3bf465cb5e2ec9c6251ece5/users/388a1e08eb1a4aa7b41756b0664a0725/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:31.216881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ea060-41e2-43de-83fd-510d4ff8e11b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:31.218346 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-015ea060-41e2-43de-83fd-510d4ff8e11b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:31.219646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dc357-e70d-43d7-89cf-d823f7b2cfa5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:31.254639 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:31.301982 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dc357-e70d-43d7-89cf-d823f7b2cfa5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:31.307653 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df58a6f-df10-4df5-8537-61ccbd976a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:31.308344 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df58a6f-df10-4df5-8537-61ccbd976a31 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:31.309509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df58a6f-df10-4df5-8537-61ccbd976a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:31.309942 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df58a6f-df10-4df5-8537-61ccbd976a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:31.310571 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df58a6f-df10-4df5-8537-61ccbd976a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:31.312859 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df58a6f-df10-4df5-8537-61ccbd976a31 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:31.323980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ea060-41e2-43de-83fd-510d4ff8e11b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:31.328371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1419/2828] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:30:31.329423 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df58a6f-df10-4df5-8537-61ccbd976a31 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:31.340515 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df58a6f-df10-4df5-8537-61ccbd976a31 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:31.345763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111271fc-1bb7-46d9-ab84-147fe64792c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:31.346668 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111271fc-1bb7-46d9-ab84-147fe64792c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:31.347105 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111271fc-1bb7-46d9-ab84-147fe64792c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:31.347378 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111271fc-1bb7-46d9-ab84-147fe64792c1 admin admin] PATH_INFO: `/v3/projects/ae1cf229e3bf465cb5e2ec9c6251ece5/users/388a1e08eb1a4aa7b41756b0664a0725/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:31.349400 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111271fc-1bb7-46d9-ab84-147fe64792c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae1cf229e3bf465cb5e2ec9c6251ece5, user_id=388a1e08eb1a4aa7b41756b0664a0725, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:31.352085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111271fc-1bb7-46d9-ab84-147fe64792c1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:31.363880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1420/2829] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:31 2022] PUT /identity/v3/projects/ae1cf229e3bf465cb5e2ec9c6251ece5/users/388a1e08eb1a4aa7b41756b0664a0725/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:31.375494 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd0426-b2c5-449a-ad12-51f81ffcf132 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:31.375777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd0426-b2c5-449a-ad12-51f81ffcf132 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:31.376049 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd0426-b2c5-449a-ad12-51f81ffcf132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:31.405282 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11af270-9fe0-4549-80de-b2282a673c12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:31.405887 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11af270-9fe0-4549-80de-b2282a673c12 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:31.406162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11af270-9fe0-4549-80de-b2282a673c12 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:31.406425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11af270-9fe0-4549-80de-b2282a673c12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:31.408229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-01bd0426-b2c5-449a-ad12-51f81ffcf132 None None] MFA Rules not processed for user `388a1e08eb1a4aa7b41756b0664a0725`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:31.408365 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d11af270-9fe0-4549-80de-b2282a673c12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:31.415368 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-01bd0426-b2c5-449a-ad12-51f81ffcf132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:31.427316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11af270-9fe0-4549-80de-b2282a673c12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:31.429027 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11af270-9fe0-4549-80de-b2282a673c12 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:31.446715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1410/2830] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:30:31.500702 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1421/2831] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:30:31.510302 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11af270-9fe0-4549-80de-b2282a673c12 service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:31.511816 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d11af270-9fe0-4549-80de-b2282a673c12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:31.573481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11af270-9fe0-4549-80de-b2282a673c12 service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:31.628154 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1b840-6a8f-451d-932c-9e92ded8e9ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:31.628809 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1b840-6a8f-451d-932c-9e92ded8e9ef service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:31.629086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1b840-6a8f-451d-932c-9e92ded8e9ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:31.629359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1b840-6a8f-451d-932c-9e92ded8e9ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:31.642176 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1b840-6a8f-451d-932c-9e92ded8e9ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:31.643615 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1b840-6a8f-451d-932c-9e92ded8e9ef service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:31.654703 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1411/2832] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:30:31.703777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd0426-b2c5-449a-ad12-51f81ffcf132 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:31.705321 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-01bd0426-b2c5-449a-ad12-51f81ffcf132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:31.718359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd0426-b2c5-449a-ad12-51f81ffcf132 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:31.722608 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81625274-d9ef-4d8d-8a92-5876ff35fd7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:31.723223 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81625274-d9ef-4d8d-8a92-5876ff35fd7a service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:31.723495 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81625274-d9ef-4d8d-8a92-5876ff35fd7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:31.723763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81625274-d9ef-4d8d-8a92-5876ff35fd7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:31.770800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:31.785911 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81625274-d9ef-4d8d-8a92-5876ff35fd7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:31.787998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81625274-d9ef-4d8d-8a92-5876ff35fd7a service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:31.798322 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1422/2833] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:30:34.660481 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1b840-6a8f-451d-932c-9e92ded8e9ef service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:34.663105 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fba1b840-6a8f-451d-932c-9e92ded8e9ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:34.690157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:34.749236 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1b840-6a8f-451d-932c-9e92ded8e9ef service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:34.754316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:34.755019 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:34.755414 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:34.755766 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] PATH_INFO: `/v3/users/d52264275fdb411484cfc6b39b625d8b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:34.757438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52264275fdb411484cfc6b39b625d8b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:34.759423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:34.814871 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52264275fdb411484cfc6b39b625d8b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:34.815603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52264275fdb411484cfc6b39b625d8b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:34.823627 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52264275fdb411484cfc6b39b625d8b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:34.838032 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52264275fdb411484cfc6b39b625d8b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:34.839072 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52264275fdb411484cfc6b39b625d8b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:34.854086 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1412/2834] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:30:34 2022] DELETE /identity/v3/users/d52264275fdb411484cfc6b39b625d8b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:35.069366 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81625274-d9ef-4d8d-8a92-5876ff35fd7a service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:35.086291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81625274-d9ef-4d8d-8a92-5876ff35fd7a service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:35.180603 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:35.181551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:35.182009 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:35.182433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] PATH_INFO: `/v3/projects/ca88b1d59a924130a3fa43b0b256c33e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:35.184193 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca88b1d59a924130a3fa43b0b256c33e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:35.186981 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:35.252917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca88b1d59a924130a3fa43b0b256c33e', 'request_id': 'req-a1f929ff-11b4-48f3-a103-58acb2fb9a86'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.260476 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca88b1d59a924130a3fa43b0b256c33e', 'request_id': 'req-a1f929ff-11b4-48f3-a103-58acb2fb9a86'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.270059 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca88b1d59a924130a3fa43b0b256c33e', 'request_id': 'req-a1f929ff-11b4-48f3-a103-58acb2fb9a86'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.284701 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] The token cache is being invalidate because project ca88b1d59a924130a3fa43b0b256c33e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:30:35.285302 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.286129 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.288618 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1423/2835] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:30:35 2022] DELETE /identity/v3/projects/ca88b1d59a924130a3fa43b0b256c33e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:35.292611 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:35.294158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:35.321969 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eaea70-a3b1-471e-8eac-93579ab8db14 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:35.389141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfeb2da-26bd-4696-a7ec-8c332dd69f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:35.389644 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfeb2da-26bd-4696-a7ec-8c332dd69f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:35.389762 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfeb2da-26bd-4696-a7ec-8c332dd69f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:35.390011 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfeb2da-26bd-4696-a7ec-8c332dd69f7f admin admin] PATH_INFO: `/v3/users/bdbc2fa907eb42509bf2517c648718bc` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:35.391162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfeb2da-26bd-4696-a7ec-8c332dd69f7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdbc2fa907eb42509bf2517c648718bc)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:35.393591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfeb2da-26bd-4696-a7ec-8c332dd69f7f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:35.412601 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:35.415433 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:35.430815 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f929ff-11b4-48f3-a103-58acb2fb9a86 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:35.436148 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:35.436926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:35.437203 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:35.437484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] PATH_INFO: `/v3/users/9901966e03ae4edda0042fc11d09d545` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:35.438873 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9901966e03ae4edda0042fc11d09d545)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:35.441050 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:35.445175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:35.457200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5bfeb2da-26bd-4696-a7ec-8c332dd69f7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdbc2fa907eb42509bf2517c648718bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.458228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5bfeb2da-26bd-4696-a7ec-8c332dd69f7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdbc2fa907eb42509bf2517c648718bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.466591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5bfeb2da-26bd-4696-a7ec-8c332dd69f7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdbc2fa907eb42509bf2517c648718bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.487573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5bfeb2da-26bd-4696-a7ec-8c332dd69f7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdbc2fa907eb42509bf2517c648718bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.488379 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5bfeb2da-26bd-4696-a7ec-8c332dd69f7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdbc2fa907eb42509bf2517c648718bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.502274 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9901966e03ae4edda0042fc11d09d545'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.503193 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9901966e03ae4edda0042fc11d09d545'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.505551 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1413/2836] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:30:35 2022] DELETE /identity/v3/users/bdbc2fa907eb42509bf2517c648718bc => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:35.511559 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9901966e03ae4edda0042fc11d09d545'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.529841 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:35.532425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9901966e03ae4edda0042fc11d09d545'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.533540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9901966e03ae4edda0042fc11d09d545'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.555362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1424/2837] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:30:35 2022] DELETE /identity/v3/users/9901966e03ae4edda0042fc11d09d545 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:30:35.562255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d509426-fb01-4c6a-b36c-9172ab7ef1c1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:35.562255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d509426-fb01-4c6a-b36c-9172ab7ef1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:35.562255 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d509426-fb01-4c6a-b36c-9172ab7ef1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:35.581117 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7d509426-fb01-4c6a-b36c-9172ab7ef1c1 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:35.582369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7d509426-fb01-4c6a-b36c-9172ab7ef1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:35.673454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1414/2838] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:35.686686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:35.688262 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:35.715039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d509426-fb01-4c6a-b36c-9172ab7ef1c1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:35.716417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7d509426-fb01-4c6a-b36c-9172ab7ef1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:35.733749 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d509426-fb01-4c6a-b36c-9172ab7ef1c1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:35.739165 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:35.740995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:35.741447 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:35.741850 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] PATH_INFO: `/v3/projects/9ac7b59cce7e4f9fa9b3f04eb96acc9e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:35.743251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ac7b59cce7e4f9fa9b3f04eb96acc9e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:35.745551 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:35.767498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798502c-39f0-4884-a2fe-ff6e6cf9f16f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:35.771666 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681483ae-a6db-4fb3-91aa-d085c9c52023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:35.772371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681483ae-a6db-4fb3-91aa-d085c9c52023 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:35.772647 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681483ae-a6db-4fb3-91aa-d085c9c52023 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:35.772940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681483ae-a6db-4fb3-91aa-d085c9c52023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:35.782835 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681483ae-a6db-4fb3-91aa-d085c9c52023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:35.784301 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681483ae-a6db-4fb3-91aa-d085c9c52023 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:35.795256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1425/2839] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 21:30:35.808421 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ac7b59cce7e4f9fa9b3f04eb96acc9e', 'request_id': 'req-bfde4e19-2b06-4f3f-ba70-c66d44a13340'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.818869 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ac7b59cce7e4f9fa9b3f04eb96acc9e', 'request_id': 'req-bfde4e19-2b06-4f3f-ba70-c66d44a13340'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.826153 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ac7b59cce7e4f9fa9b3f04eb96acc9e', 'request_id': 'req-bfde4e19-2b06-4f3f-ba70-c66d44a13340'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.838714 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] The token cache is being invalidate because project 9ac7b59cce7e4f9fa9b3f04eb96acc9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:30:35.838999 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.839664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:35.841585 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1415/2840] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:30:35 2022] DELETE /identity/v3/projects/9ac7b59cce7e4f9fa9b3f04eb96acc9e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:30:36.134008 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681483ae-a6db-4fb3-91aa-d085c9c52023 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:36.135532 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-681483ae-a6db-4fb3-91aa-d085c9c52023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:36.149034 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681483ae-a6db-4fb3-91aa-d085c9c52023 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:36.208013 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:36.208703 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:36.209014 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:36.209286 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] PATH_INFO: `/v3/projects/a2104705c6514c9e94c10fedf14f31ee` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:36.210523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2104705c6514c9e94c10fedf14f31ee)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:36.212402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:36.383500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2104705c6514c9e94c10fedf14f31ee', 'request_id': 'req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:36.390098 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2104705c6514c9e94c10fedf14f31ee', 'request_id': 'req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:36.395778 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2104705c6514c9e94c10fedf14f31ee', 'request_id': 'req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:36.412373 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] The token cache is being invalidate because project a2104705c6514c9e94c10fedf14f31ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:30:36.412795 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:36.413402 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:36.414985 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1426/2841] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:30:36 2022] DELETE /identity/v3/projects/a2104705c6514c9e94c10fedf14f31ee => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:36.419296 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:36.420769 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:36.434794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde4e19-2b06-4f3f-ba70-c66d44a13340 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:36.468976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:36.493951 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83aeda-4f14-4104-886d-6108427d4d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:36.494591 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83aeda-4f14-4104-886d-6108427d4d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:36.494868 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83aeda-4f14-4104-886d-6108427d4d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:36.495138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83aeda-4f14-4104-886d-6108427d4d25 admin admin] PATH_INFO: `/v3/users/388a1e08eb1a4aa7b41756b0664a0725` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:36.496248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83aeda-4f14-4104-886d-6108427d4d25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=388a1e08eb1a4aa7b41756b0664a0725)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:36.498208 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83aeda-4f14-4104-886d-6108427d4d25 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:36.547553 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fb83aeda-4f14-4104-886d-6108427d4d25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388a1e08eb1a4aa7b41756b0664a0725'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:36.548192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fb83aeda-4f14-4104-886d-6108427d4d25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388a1e08eb1a4aa7b41756b0664a0725'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:36.555134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fb83aeda-4f14-4104-886d-6108427d4d25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388a1e08eb1a4aa7b41756b0664a0725'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:36.566999 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fb83aeda-4f14-4104-886d-6108427d4d25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388a1e08eb1a4aa7b41756b0664a0725'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:36.567649 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fb83aeda-4f14-4104-886d-6108427d4d25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388a1e08eb1a4aa7b41756b0664a0725'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:36.582261 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1416/2842] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:30:36 2022] DELETE /identity/v3/users/388a1e08eb1a4aa7b41756b0664a0725 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:36.869143 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:36.870736 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:36.887888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dc212-a3fb-4959-a1f0-92b82d4d6e62 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:36.906304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:36.956523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:36.957191 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:36.957464 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:36.957729 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e admin admin] PATH_INFO: `/v3/projects/ae1cf229e3bf465cb5e2ec9c6251ece5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:36.958888 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae1cf229e3bf465cb5e2ec9c6251ece5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:36.960825 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:37.009484 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae1cf229e3bf465cb5e2ec9c6251ece5', 'request_id': 'req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:37.018017 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae1cf229e3bf465cb5e2ec9c6251ece5', 'request_id': 'req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:37.024763 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae1cf229e3bf465cb5e2ec9c6251ece5', 'request_id': 'req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:37.040010 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e admin admin] The token cache is being invalidate because project ae1cf229e3bf465cb5e2ec9c6251ece5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:30:37.040299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:37.040978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-efa7b1e3-4068-4e07-9063-7d9fe032ab9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:37.042499 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1427/2843] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:30:36 2022] DELETE /identity/v3/projects/ae1cf229e3bf465cb5e2ec9c6251ece5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:37.047719 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111e2c9-b488-48ca-b6b8-7b2d47ec5206 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:37.048209 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111e2c9-b488-48ca-b6b8-7b2d47ec5206 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:37.048331 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111e2c9-b488-48ca-b6b8-7b2d47ec5206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:37.069688 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e111e2c9-b488-48ca-b6b8-7b2d47ec5206 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:37.070916 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e111e2c9-b488-48ca-b6b8-7b2d47ec5206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:37.141611 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1417/2844] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:30:37.149928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4a80a-3a6d-46ac-a6c1-32add97feb49 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:37.150488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4a80a-3a6d-46ac-a6c1-32add97feb49 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:37.150488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4a80a-3a6d-46ac-a6c1-32add97feb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:37.171902 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1ee4a80a-3a6d-46ac-a6c1-32add97feb49 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:37.173182 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1ee4a80a-3a6d-46ac-a6c1-32add97feb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:37.182296 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1428/2845] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:37.186663 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111e2c9-b488-48ca-b6b8-7b2d47ec5206 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:37.197284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111e2c9-b488-48ca-b6b8-7b2d47ec5206 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:37.203762 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc694e19-226f-49ae-9cd8-ee05b962569c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:37.204716 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc694e19-226f-49ae-9cd8-ee05b962569c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:37.205069 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc694e19-226f-49ae-9cd8-ee05b962569c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:37.205417 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc694e19-226f-49ae-9cd8-ee05b962569c admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:37.206355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc694e19-226f-49ae-9cd8-ee05b962569c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:30:37.206773 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc694e19-226f-49ae-9cd8-ee05b962569c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:37.209285 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc694e19-226f-49ae-9cd8-ee05b962569c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:37.221973 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1418/2846] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:30:37.226513 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4a80a-3a6d-46ac-a6c1-32add97feb49 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:37.236295 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4a80a-3a6d-46ac-a6c1-32add97feb49 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:37.241737 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36c86d-9c63-44be-982b-b50216a4a1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:37.242362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36c86d-9c63-44be-982b-b50216a4a1c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:37.242637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36c86d-9c63-44be-982b-b50216a4a1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:37.242905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36c86d-9c63-44be-982b-b50216a4a1c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:37.243551 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36c86d-9c63-44be-982b-b50216a4a1c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:37.245543 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36c86d-9c63-44be-982b-b50216a4a1c5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:37.276749 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1429/2847] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:30:37 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:37.281226 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc694e19-226f-49ae-9cd8-ee05b962569c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:37.292439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc694e19-226f-49ae-9cd8-ee05b962569c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:37.341159 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:37.359849 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790d7e5-9f08-4f92-b4f5-b0818017b333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:37.360466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790d7e5-9f08-4f92-b4f5-b0818017b333 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:37.360780 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790d7e5-9f08-4f92-b4f5-b0818017b333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:37.361051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790d7e5-9f08-4f92-b4f5-b0818017b333 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:37.361715 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790d7e5-9f08-4f92-b4f5-b0818017b333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:37.363948 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790d7e5-9f08-4f92-b4f5-b0818017b333 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:37.389555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1419/2848] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:30:37 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:37.393881 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36c86d-9c63-44be-982b-b50216a4a1c5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:37.404780 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36c86d-9c63-44be-982b-b50216a4a1c5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:37.410440 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d8f22-d403-4a26-8974-7f13084af910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:37.411045 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d8f22-d403-4a26-8974-7f13084af910 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:37.411318 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d8f22-d403-4a26-8974-7f13084af910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:37.411936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d8f22-d403-4a26-8974-7f13084af910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:37.412585 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d8f22-d403-4a26-8974-7f13084af910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:37.414256 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d8f22-d403-4a26-8974-7f13084af910 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:37.424640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1430/2849] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:37.429352 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790d7e5-9f08-4f92-b4f5-b0818017b333 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:37.439938 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790d7e5-9f08-4f92-b4f5-b0818017b333 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:37.445949 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043cbfb-fec6-4cf3-8158-766d7fa352e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:37.446529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043cbfb-fec6-4cf3-8158-766d7fa352e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:37.446805 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043cbfb-fec6-4cf3-8158-766d7fa352e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:37.447062 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043cbfb-fec6-4cf3-8158-766d7fa352e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:37.447708 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043cbfb-fec6-4cf3-8158-766d7fa352e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:37.449361 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043cbfb-fec6-4cf3-8158-766d7fa352e1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:37.461530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1420/2850] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:37.465978 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d8f22-d403-4a26-8974-7f13084af910 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:37.476631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d8f22-d403-4a26-8974-7f13084af910 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:37.481950 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a139a-a8e1-4bd7-ba03-74916d452e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:37.482856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a139a-a8e1-4bd7-ba03-74916d452e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:37.482856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a139a-a8e1-4bd7-ba03-74916d452e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:37.483173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a139a-a8e1-4bd7-ba03-74916d452e01 admin admin] PATH_INFO: `/v3/projects/46324e2680d04163881f257e744ed821/users/127f321ed8d547fbbc113ba342c82139/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:37.502448 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a139a-a8e1-4bd7-ba03-74916d452e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46324e2680d04163881f257e744ed821, user_id=127f321ed8d547fbbc113ba342c82139, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:37.506727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a139a-a8e1-4bd7-ba03-74916d452e01 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:37.525305 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1431/2851] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:37 2022] PUT /identity/v3/projects/46324e2680d04163881f257e744ed821/users/127f321ed8d547fbbc113ba342c82139/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:37.541707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd2000-71ea-411a-a9f3-976cb9850ef8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:37.541887 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd2000-71ea-411a-a9f3-976cb9850ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:37.542178 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd2000-71ea-411a-a9f3-976cb9850ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:37.571377 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e1fd2000-71ea-411a-a9f3-976cb9850ef8 None None] MFA Rules not processed for user `127f321ed8d547fbbc113ba342c82139`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:37.572621 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e1fd2000-71ea-411a-a9f3-976cb9850ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:37.640877 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1421/2852] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:37.645689 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a139a-a8e1-4bd7-ba03-74916d452e01 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:37.700952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:37.719213 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a139a-a8e1-4bd7-ba03-74916d452e01 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:37.724466 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bc642-ad56-4623-85fc-3fb40bb52cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:37.725312 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bc642-ad56-4623-85fc-3fb40bb52cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:37.725590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bc642-ad56-4623-85fc-3fb40bb52cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:37.725879 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bc642-ad56-4623-85fc-3fb40bb52cf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:37.726509 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bc642-ad56-4623-85fc-3fb40bb52cf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:37.728491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bc642-ad56-4623-85fc-3fb40bb52cf8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:37.757403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1432/2853] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:30:37 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:37.761701 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd2000-71ea-411a-a9f3-976cb9850ef8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:37.773928 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd2000-71ea-411a-a9f3-976cb9850ef8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:37.833791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d1f67-3059-456e-b921-63f5dc01ba60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:37.834313 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d1f67-3059-456e-b921-63f5dc01ba60 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:37.834640 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d1f67-3059-456e-b921-63f5dc01ba60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:37.834903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d1f67-3059-456e-b921-63f5dc01ba60 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:37.835523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d1f67-3059-456e-b921-63f5dc01ba60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:37.837466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d1f67-3059-456e-b921-63f5dc01ba60 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:37.863834 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1422/2854] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:30:37 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:37.868157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bc642-ad56-4623-85fc-3fb40bb52cf8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:37.882641 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bc642-ad56-4623-85fc-3fb40bb52cf8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:37.888578 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2579f9-4c59-4c30-87dd-a346aba741e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:37.889219 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2579f9-4c59-4c30-87dd-a346aba741e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:37.889493 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2579f9-4c59-4c30-87dd-a346aba741e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:37.889761 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2579f9-4c59-4c30-87dd-a346aba741e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:37.890393 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2579f9-4c59-4c30-87dd-a346aba741e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:37.892002 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2579f9-4c59-4c30-87dd-a346aba741e4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:37.904020 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1433/2855] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:37.908427 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d1f67-3059-456e-b921-63f5dc01ba60 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:37.919293 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d1f67-3059-456e-b921-63f5dc01ba60 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:37.924300 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a80861-1cba-43a0-a1c6-ace1bb6e799c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:37.924983 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a80861-1cba-43a0-a1c6-ace1bb6e799c admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:37.925261 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a80861-1cba-43a0-a1c6-ace1bb6e799c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:37.925531 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a80861-1cba-43a0-a1c6-ace1bb6e799c admin admin] PATH_INFO: `/v3/projects/9fb2482bc86043b8acf157dec3cfd581/users/2686d9031ea14186ae08125ab5b6e605/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:37.941188 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a80861-1cba-43a0-a1c6-ace1bb6e799c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb2482bc86043b8acf157dec3cfd581, user_id=2686d9031ea14186ae08125ab5b6e605, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:37.943970 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a80861-1cba-43a0-a1c6-ace1bb6e799c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:37.958527 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1423/2856] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:37 2022] PUT /identity/v3/projects/9fb2482bc86043b8acf157dec3cfd581/users/2686d9031ea14186ae08125ab5b6e605/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:37.962727 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2579f9-4c59-4c30-87dd-a346aba741e4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:37.972947 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2579f9-4c59-4c30-87dd-a346aba741e4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:38.030940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c13ac-236b-4763-bce0-59fb27045477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:38.031546 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c13ac-236b-4763-bce0-59fb27045477 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:38.031823 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c13ac-236b-4763-bce0-59fb27045477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:38.032088 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c13ac-236b-4763-bce0-59fb27045477 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:38.032732 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c13ac-236b-4763-bce0-59fb27045477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:38.034337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c13ac-236b-4763-bce0-59fb27045477 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:38.044239 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1434/2857] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:38.048613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a80861-1cba-43a0-a1c6-ace1bb6e799c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:38.059369 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a80861-1cba-43a0-a1c6-ace1bb6e799c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:38.065407 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33722801-76c4-4b9a-bdb5-b3f83eaa5794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:38.066059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33722801-76c4-4b9a-bdb5-b3f83eaa5794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:38.066346 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33722801-76c4-4b9a-bdb5-b3f83eaa5794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:38.066615 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33722801-76c4-4b9a-bdb5-b3f83eaa5794 admin admin] PATH_INFO: `/v3/projects/9fb2482bc86043b8acf157dec3cfd581/users/2686d9031ea14186ae08125ab5b6e605/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:38.068676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33722801-76c4-4b9a-bdb5-b3f83eaa5794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb2482bc86043b8acf157dec3cfd581, user_id=2686d9031ea14186ae08125ab5b6e605, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:38.071340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33722801-76c4-4b9a-bdb5-b3f83eaa5794 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:38.082619 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1424/2858] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:38 2022] PUT /identity/v3/projects/9fb2482bc86043b8acf157dec3cfd581/users/2686d9031ea14186ae08125ab5b6e605/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:38.150912 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611d46f-7e3e-4e8c-a8b2-c098627bf91d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:38.151477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611d46f-7e3e-4e8c-a8b2-c098627bf91d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:38.151477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611d46f-7e3e-4e8c-a8b2-c098627bf91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:38.196689 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5611d46f-7e3e-4e8c-a8b2-c098627bf91d None None] MFA Rules not processed for user `2686d9031ea14186ae08125ab5b6e605`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:38.197944 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5611d46f-7e3e-4e8c-a8b2-c098627bf91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:38.275173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1435/2859] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:38.290549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33722801-76c4-4b9a-bdb5-b3f83eaa5794 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:38.292511 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-33722801-76c4-4b9a-bdb5-b3f83eaa5794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:38.299707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:38.356835 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33722801-76c4-4b9a-bdb5-b3f83eaa5794 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:38.360647 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ffbf5-bf63-4f10-a706-cf7e2eff3f3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:38.361249 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ffbf5-bf63-4f10-a706-cf7e2eff3f3a service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:38.361523 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ffbf5-bf63-4f10-a706-cf7e2eff3f3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:38.361793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ffbf5-bf63-4f10-a706-cf7e2eff3f3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:38.371078 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ffbf5-bf63-4f10-a706-cf7e2eff3f3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:38.372583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ffbf5-bf63-4f10-a706-cf7e2eff3f3a service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:38.382777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1425/2860] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:30:38.416238 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611d46f-7e3e-4e8c-a8b2-c098627bf91d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:38.465297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:38.484231 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611d46f-7e3e-4e8c-a8b2-c098627bf91d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:38.489323 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:38.489955 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:38.490229 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:38.490500 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] PATH_INFO: `/v3/users/127f321ed8d547fbbc113ba342c82139` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:38.491686 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=127f321ed8d547fbbc113ba342c82139)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:38.493672 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:38.547878 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127f321ed8d547fbbc113ba342c82139'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:38.548547 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127f321ed8d547fbbc113ba342c82139'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:38.556004 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127f321ed8d547fbbc113ba342c82139'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:38.569512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127f321ed8d547fbbc113ba342c82139'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:38.570249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127f321ed8d547fbbc113ba342c82139'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:38.585614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1436/2861] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:30:38 2022] DELETE /identity/v3/users/127f321ed8d547fbbc113ba342c82139 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:38.590479 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefceb10-9f37-40f8-9ce7-341149b9e1b7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:38.590804 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefceb10-9f37-40f8-9ce7-341149b9e1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:38.591742 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefceb10-9f37-40f8-9ce7-341149b9e1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:38.610570 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-aefceb10-9f37-40f8-9ce7-341149b9e1b7 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:38.611777 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aefceb10-9f37-40f8-9ce7-341149b9e1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:38.679397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1426/2862] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:38.689605 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:38.691128 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:38.753980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d15e0c-f79d-4eae-a2b7-e762ba173a2b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:38.763581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e81144-c854-47d3-877d-9dc32fdfc470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:38.763581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e81144-c854-47d3-877d-9dc32fdfc470 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:38.763581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e81144-c854-47d3-877d-9dc32fdfc470 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:38.763581 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e81144-c854-47d3-877d-9dc32fdfc470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:38.772425 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e81144-c854-47d3-877d-9dc32fdfc470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:38.773900 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e81144-c854-47d3-877d-9dc32fdfc470 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:38.784137 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1437/2863] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:30:39.084354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefceb10-9f37-40f8-9ce7-341149b9e1b7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:39.085995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aefceb10-9f37-40f8-9ce7-341149b9e1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:39.098705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefceb10-9f37-40f8-9ce7-341149b9e1b7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:39.105244 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:39.105881 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:39.106157 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:39.106555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] PATH_INFO: `/v3/projects/46324e2680d04163881f257e744ed821` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:39.107911 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46324e2680d04163881f257e744ed821)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:39.109869 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:39.167387 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46324e2680d04163881f257e744ed821', 'request_id': 'req-d554d996-8f19-4cd6-8b78-ba5e931c73a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.177212 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46324e2680d04163881f257e744ed821', 'request_id': 'req-d554d996-8f19-4cd6-8b78-ba5e931c73a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.184367 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46324e2680d04163881f257e744ed821', 'request_id': 'req-d554d996-8f19-4cd6-8b78-ba5e931c73a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.195251 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] The token cache is being invalidate because project 46324e2680d04163881f257e744ed821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:30:39.195532 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.196201 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.197791 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1427/2864] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:30:39 2022] DELETE /identity/v3/projects/46324e2680d04163881f257e744ed821 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:39.201980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e81144-c854-47d3-877d-9dc32fdfc470 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:39.211883 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e81144-c854-47d3-877d-9dc32fdfc470 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:39.269381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca0dc6-f8a4-4373-9edd-0e76eb06c387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:39.270013 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca0dc6-f8a4-4373-9edd-0e76eb06c387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:39.270288 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca0dc6-f8a4-4373-9edd-0e76eb06c387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:39.270557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca0dc6-f8a4-4373-9edd-0e76eb06c387 admin admin] PATH_INFO: `/v3/users/2686d9031ea14186ae08125ab5b6e605` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:39.271716 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca0dc6-f8a4-4373-9edd-0e76eb06c387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2686d9031ea14186ae08125ab5b6e605)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:39.273669 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca0dc6-f8a4-4373-9edd-0e76eb06c387 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:39.278339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:39.324214 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41ca0dc6-f8a4-4373-9edd-0e76eb06c387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2686d9031ea14186ae08125ab5b6e605'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.324905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41ca0dc6-f8a4-4373-9edd-0e76eb06c387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2686d9031ea14186ae08125ab5b6e605'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.331850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41ca0dc6-f8a4-4373-9edd-0e76eb06c387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2686d9031ea14186ae08125ab5b6e605'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.348610 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41ca0dc6-f8a4-4373-9edd-0e76eb06c387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2686d9031ea14186ae08125ab5b6e605'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.349733 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41ca0dc6-f8a4-4373-9edd-0e76eb06c387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2686d9031ea14186ae08125ab5b6e605'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.366265 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1438/2865] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:30:39 2022] DELETE /identity/v3/users/2686d9031ea14186ae08125ab5b6e605 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:39.650286 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:39.651823 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:39.667168 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554d996-8f19-4cd6-8b78-ba5e931c73a1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:39.700158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:39.730116 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1982cc-0152-4f26-8c13-94d73c085b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:39.730658 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1982cc-0152-4f26-8c13-94d73c085b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:39.730930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1982cc-0152-4f26-8c13-94d73c085b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:39.731195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1982cc-0152-4f26-8c13-94d73c085b12 admin admin] PATH_INFO: `/v3/projects/9fb2482bc86043b8acf157dec3cfd581` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:39.732318 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1982cc-0152-4f26-8c13-94d73c085b12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb2482bc86043b8acf157dec3cfd581)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:39.734270 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1982cc-0152-4f26-8c13-94d73c085b12 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:39.781058 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5b1982cc-0152-4f26-8c13-94d73c085b12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb2482bc86043b8acf157dec3cfd581', 'request_id': 'req-5b1982cc-0152-4f26-8c13-94d73c085b12'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.789817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5b1982cc-0152-4f26-8c13-94d73c085b12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb2482bc86043b8acf157dec3cfd581', 'request_id': 'req-5b1982cc-0152-4f26-8c13-94d73c085b12'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.796490 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5b1982cc-0152-4f26-8c13-94d73c085b12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb2482bc86043b8acf157dec3cfd581', 'request_id': 'req-5b1982cc-0152-4f26-8c13-94d73c085b12'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.807013 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5b1982cc-0152-4f26-8c13-94d73c085b12 admin admin] The token cache is being invalidate because project 9fb2482bc86043b8acf157dec3cfd581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:30:39.807301 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5b1982cc-0152-4f26-8c13-94d73c085b12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.807965 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5b1982cc-0152-4f26-8c13-94d73c085b12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:39.809547 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1428/2866] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:30:39 2022] DELETE /identity/v3/projects/9fb2482bc86043b8acf157dec3cfd581 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:39.814808 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e1237-978e-4c07-b2a5-bcaffa8093fd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:39.815113 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e1237-978e-4c07-b2a5-bcaffa8093fd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:39.815376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e1237-978e-4c07-b2a5-bcaffa8093fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:39.836777 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fc2e1237-978e-4c07-b2a5-bcaffa8093fd None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:39.838025 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fc2e1237-978e-4c07-b2a5-bcaffa8093fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:39.898856 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1439/2867] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:30:39.907794 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed5f52-127f-4a2b-9029-5b997a64de1e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:39.908166 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed5f52-127f-4a2b-9029-5b997a64de1e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:39.908536 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed5f52-127f-4a2b-9029-5b997a64de1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:39.934785 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4eed5f52-127f-4a2b-9029-5b997a64de1e None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:39.936281 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4eed5f52-127f-4a2b-9029-5b997a64de1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:39.946901 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1429/2868] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:39.953369 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e1237-978e-4c07-b2a5-bcaffa8093fd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:39.968514 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e1237-978e-4c07-b2a5-bcaffa8093fd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:39.976579 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31113662-1e6e-47fa-9ef8-cc48838840c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:39.977217 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31113662-1e6e-47fa-9ef8-cc48838840c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:39.977492 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31113662-1e6e-47fa-9ef8-cc48838840c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:39.977752 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31113662-1e6e-47fa-9ef8-cc48838840c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:39.978390 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31113662-1e6e-47fa-9ef8-cc48838840c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:30:39.978685 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31113662-1e6e-47fa-9ef8-cc48838840c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:39.980511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31113662-1e6e-47fa-9ef8-cc48838840c3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:39.992354 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1440/2869] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:30:39.996826 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed5f52-127f-4a2b-9029-5b997a64de1e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:40.007084 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed5f52-127f-4a2b-9029-5b997a64de1e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:40.012420 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2ac6c-ccad-4b73-86c7-688d02250155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:40.013065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2ac6c-ccad-4b73-86c7-688d02250155 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:40.013345 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2ac6c-ccad-4b73-86c7-688d02250155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:40.013613 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2ac6c-ccad-4b73-86c7-688d02250155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:40.014248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2ac6c-ccad-4b73-86c7-688d02250155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:40.016151 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2ac6c-ccad-4b73-86c7-688d02250155 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:40.048199 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1430/2870] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:30:39 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:40.052458 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31113662-1e6e-47fa-9ef8-cc48838840c3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:40.064643 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31113662-1e6e-47fa-9ef8-cc48838840c3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:40.126917 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a9561-842c-4bb3-81e9-3ddae30d50bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:40.127900 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a9561-842c-4bb3-81e9-3ddae30d50bc admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:40.128337 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a9561-842c-4bb3-81e9-3ddae30d50bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:40.128811 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a9561-842c-4bb3-81e9-3ddae30d50bc admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:40.129870 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a9561-842c-4bb3-81e9-3ddae30d50bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:40.133001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a9561-842c-4bb3-81e9-3ddae30d50bc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:40.154974 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:40.162649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1441/2871] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:30:40 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:40.167765 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2ac6c-ccad-4b73-86c7-688d02250155 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:40.179020 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2ac6c-ccad-4b73-86c7-688d02250155 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:40.184906 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47516df-f5c7-440c-8981-cc2a0175a9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:40.185499 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47516df-f5c7-440c-8981-cc2a0175a9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:40.185796 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47516df-f5c7-440c-8981-cc2a0175a9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:40.186059 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47516df-f5c7-440c-8981-cc2a0175a9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:40.186691 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47516df-f5c7-440c-8981-cc2a0175a9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:40.188316 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47516df-f5c7-440c-8981-cc2a0175a9b4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:40.199272 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1431/2872] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:40.203561 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a9561-842c-4bb3-81e9-3ddae30d50bc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:40.214518 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a9561-842c-4bb3-81e9-3ddae30d50bc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:40.219646 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8097d9d-0840-476b-bb15-85549b2209fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:40.220255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8097d9d-0840-476b-bb15-85549b2209fb admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:40.220536 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8097d9d-0840-476b-bb15-85549b2209fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:40.220848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8097d9d-0840-476b-bb15-85549b2209fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:40.221498 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8097d9d-0840-476b-bb15-85549b2209fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:40.223104 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8097d9d-0840-476b-bb15-85549b2209fb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:40.234294 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1442/2873] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 21:30:40.238555 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47516df-f5c7-440c-8981-cc2a0175a9b4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:40.249196 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47516df-f5c7-440c-8981-cc2a0175a9b4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:40.254147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edadb83-af7d-41e0-a769-8922846da51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:40.254798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edadb83-af7d-41e0-a769-8922846da51e admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:40.255079 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edadb83-af7d-41e0-a769-8922846da51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:40.255354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edadb83-af7d-41e0-a769-8922846da51e admin admin] PATH_INFO: `/v3/projects/8674ffdcb49c431aa489b50e0ad0cf75/users/eb4d5a1badca4a58bcb52488ba61b1a7/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:40.270882 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edadb83-af7d-41e0-a769-8922846da51e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8674ffdcb49c431aa489b50e0ad0cf75, user_id=eb4d5a1badca4a58bcb52488ba61b1a7, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:40.273664 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edadb83-af7d-41e0-a769-8922846da51e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:40.290825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1432/2874] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:40 2022] PUT /identity/v3/projects/8674ffdcb49c431aa489b50e0ad0cf75/users/eb4d5a1badca4a58bcb52488ba61b1a7/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:40.301867 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1d011-2646-47f6-83b8-9e28e686e602 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:40.302157 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1d011-2646-47f6-83b8-9e28e686e602 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:40.302441 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1d011-2646-47f6-83b8-9e28e686e602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:40.337289 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9ae1d011-2646-47f6-83b8-9e28e686e602 None None] MFA Rules not processed for user `eb4d5a1badca4a58bcb52488ba61b1a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:40.338508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9ae1d011-2646-47f6-83b8-9e28e686e602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:40.406467 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1443/2875] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:40.411397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edadb83-af7d-41e0-a769-8922846da51e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:40.476354 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:40.480887 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edadb83-af7d-41e0-a769-8922846da51e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:40.486972 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2275b-4d94-44f6-9965-6cd036d68d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:40.487529 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2275b-4d94-44f6-9965-6cd036d68d72 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:40.487856 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2275b-4d94-44f6-9965-6cd036d68d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:40.488138 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2275b-4d94-44f6-9965-6cd036d68d72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:40.489062 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2275b-4d94-44f6-9965-6cd036d68d72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:40.491496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2275b-4d94-44f6-9965-6cd036d68d72 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:40.524113 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1433/2876] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:30:40 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:40.529228 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1d011-2646-47f6-83b8-9e28e686e602 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:40.541887 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1d011-2646-47f6-83b8-9e28e686e602 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:40.605842 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37735954-a072-4239-9dce-338b4c2591e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:40.606684 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37735954-a072-4239-9dce-338b4c2591e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:40.607036 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37735954-a072-4239-9dce-338b4c2591e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:40.607367 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37735954-a072-4239-9dce-338b4c2591e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:40.608306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37735954-a072-4239-9dce-338b4c2591e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:40.611320 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37735954-a072-4239-9dce-338b4c2591e2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:40.642596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1444/2877] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:30:40 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:40.646811 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2275b-4d94-44f6-9965-6cd036d68d72 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:40.659110 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2275b-4d94-44f6-9965-6cd036d68d72 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:40.665890 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65a777-e260-4017-a4ff-05c0315c1214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:40.666487 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65a777-e260-4017-a4ff-05c0315c1214 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:40.666761 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65a777-e260-4017-a4ff-05c0315c1214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:40.667036 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65a777-e260-4017-a4ff-05c0315c1214 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:40.667659 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65a777-e260-4017-a4ff-05c0315c1214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:40.669351 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65a777-e260-4017-a4ff-05c0315c1214 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:40.681187 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1434/2878] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:40.685798 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37735954-a072-4239-9dce-338b4c2591e2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:40.696640 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37735954-a072-4239-9dce-338b4c2591e2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:40.701879 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf13130-27b7-4c8f-ba73-d440e1cf227d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:40.702540 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf13130-27b7-4c8f-ba73-d440e1cf227d admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:40.702819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf13130-27b7-4c8f-ba73-d440e1cf227d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:40.703095 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf13130-27b7-4c8f-ba73-d440e1cf227d admin admin] PATH_INFO: `/v3/projects/c0ef9bb471e54c8db060f600df849315/users/aff941a54eea4319ad5845e22eace9a1/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:40.719597 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf13130-27b7-4c8f-ba73-d440e1cf227d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ef9bb471e54c8db060f600df849315, user_id=aff941a54eea4319ad5845e22eace9a1, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:40.722339 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf13130-27b7-4c8f-ba73-d440e1cf227d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:40.737510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1445/2879] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:40 2022] PUT /identity/v3/projects/c0ef9bb471e54c8db060f600df849315/users/aff941a54eea4319ad5845e22eace9a1/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:40.741583 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65a777-e260-4017-a4ff-05c0315c1214 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:40.752439 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65a777-e260-4017-a4ff-05c0315c1214 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:40.811654 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826bde9-b8ec-474f-a5b4-b67a242e5e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:40.812262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826bde9-b8ec-474f-a5b4-b67a242e5e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:40.812560 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826bde9-b8ec-474f-a5b4-b67a242e5e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:40.812853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826bde9-b8ec-474f-a5b4-b67a242e5e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:40.813477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826bde9-b8ec-474f-a5b4-b67a242e5e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:40.815070 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826bde9-b8ec-474f-a5b4-b67a242e5e71 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:40.824917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1435/2880] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:40.829249 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf13130-27b7-4c8f-ba73-d440e1cf227d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:40.839678 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf13130-27b7-4c8f-ba73-d440e1cf227d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:40.844431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9abd6-2dc6-452b-a681-0fdcd13423b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:40.845118 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9abd6-2dc6-452b-a681-0fdcd13423b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:40.845388 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9abd6-2dc6-452b-a681-0fdcd13423b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:40.845654 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9abd6-2dc6-452b-a681-0fdcd13423b7 admin admin] PATH_INFO: `/v3/projects/c0ef9bb471e54c8db060f600df849315/users/aff941a54eea4319ad5845e22eace9a1/roles/3b30504991194002b41f8587d0851c10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:40.847590 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9abd6-2dc6-452b-a681-0fdcd13423b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ef9bb471e54c8db060f600df849315, user_id=aff941a54eea4319ad5845e22eace9a1, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:40.850281 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9abd6-2dc6-452b-a681-0fdcd13423b7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:40.860488 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1446/2881] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:40 2022] PUT /identity/v3/projects/c0ef9bb471e54c8db060f600df849315/users/aff941a54eea4319ad5845e22eace9a1/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:40.871917 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d19426-b24c-44b1-89c9-31e70a53a5e1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:40.872200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d19426-b24c-44b1-89c9-31e70a53a5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:40.872544 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d19426-b24c-44b1-89c9-31e70a53a5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:40.902903 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f8d19426-b24c-44b1-89c9-31e70a53a5e1 None None] MFA Rules not processed for user `aff941a54eea4319ad5845e22eace9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:40.904127 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f8d19426-b24c-44b1-89c9-31e70a53a5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:40.975886 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1436/2882] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:40.987914 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9abd6-2dc6-452b-a681-0fdcd13423b7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:40.993082 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-37e9abd6-2dc6-452b-a681-0fdcd13423b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:41.028745 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:41.054874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9abd6-2dc6-452b-a681-0fdcd13423b7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:41.059124 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7be9d-ff77-46dd-8930-adfaa6c4e31f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:41.059734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7be9d-ff77-46dd-8930-adfaa6c4e31f service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:41.060011 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7be9d-ff77-46dd-8930-adfaa6c4e31f service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:41.060271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7be9d-ff77-46dd-8930-adfaa6c4e31f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:41.069371 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7be9d-ff77-46dd-8930-adfaa6c4e31f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:41.070848 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7be9d-ff77-46dd-8930-adfaa6c4e31f service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:41.080079 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1447/2883] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:30:41.156275 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d19426-b24c-44b1-89c9-31e70a53a5e1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:41.214230 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:41.226118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d19426-b24c-44b1-89c9-31e70a53a5e1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:41.231460 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:41.232121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:41.232397 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:41.232692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] PATH_INFO: `/v3/users/eb4d5a1badca4a58bcb52488ba61b1a7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:41.233853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb4d5a1badca4a58bcb52488ba61b1a7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:41.235797 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:41.286004 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb4d5a1badca4a58bcb52488ba61b1a7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:41.287145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb4d5a1badca4a58bcb52488ba61b1a7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:41.295085 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb4d5a1badca4a58bcb52488ba61b1a7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:41.307449 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb4d5a1badca4a58bcb52488ba61b1a7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:41.308104 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb4d5a1badca4a58bcb52488ba61b1a7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:41.325308 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1437/2884] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:30:41 2022] DELETE /identity/v3/users/eb4d5a1badca4a58bcb52488ba61b1a7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:41.330146 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec3372-9ea5-4308-8df8-cbe985097945 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:41.330667 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec3372-9ea5-4308-8df8-cbe985097945 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:41.330773 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec3372-9ea5-4308-8df8-cbe985097945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:41.350233 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-37ec3372-9ea5-4308-8df8-cbe985097945 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:41.351443 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-37ec3372-9ea5-4308-8df8-cbe985097945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:41.412232 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1448/2885] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:41.423294 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:41.425076 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:41.484960 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d51f3-ed7e-4bd8-8903-e62d165617f0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:41.489106 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cd3fc-3f99-473c-8d76-71ba59d6da61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:41.489707 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cd3fc-3f99-473c-8d76-71ba59d6da61 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:41.489981 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cd3fc-3f99-473c-8d76-71ba59d6da61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:41.490248 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cd3fc-3f99-473c-8d76-71ba59d6da61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:41.508039 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cd3fc-3f99-473c-8d76-71ba59d6da61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:41.511635 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cd3fc-3f99-473c-8d76-71ba59d6da61 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:41.527737 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1438/2886] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:30:41.833508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec3372-9ea5-4308-8df8-cbe985097945 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:41.835180 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-37ec3372-9ea5-4308-8df8-cbe985097945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:41.854067 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec3372-9ea5-4308-8df8-cbe985097945 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:41.860531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:41.861107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:41.861387 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:41.861615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] PATH_INFO: `/v3/projects/8674ffdcb49c431aa489b50e0ad0cf75` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:41.862861 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=8674ffdcb49c431aa489b50e0ad0cf75)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:41.865001 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:41.915523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8674ffdcb49c431aa489b50e0ad0cf75', 'request_id': 'req-82437df9-7a58-45d1-96c9-e330ac4e81ea'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:41.921632 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8674ffdcb49c431aa489b50e0ad0cf75', 'request_id': 'req-82437df9-7a58-45d1-96c9-e330ac4e81ea'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:41.927611 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8674ffdcb49c431aa489b50e0ad0cf75', 'request_id': 'req-82437df9-7a58-45d1-96c9-e330ac4e81ea'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:41.940431 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] The token cache is being invalidate because project 8674ffdcb49c431aa489b50e0ad0cf75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:30:41.940747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:41.941381 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:41.942906 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1449/2887] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:30:41 2022] DELETE /identity/v3/projects/8674ffdcb49c431aa489b50e0ad0cf75 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:41.947210 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cd3fc-3f99-473c-8d76-71ba59d6da61 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:41.957446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cd3fc-3f99-473c-8d76-71ba59d6da61 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:42.021810 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8b9bb-548b-44fe-b02f-f98f578aad9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:42.022437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8b9bb-548b-44fe-b02f-f98f578aad9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:42.022719 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8b9bb-548b-44fe-b02f-f98f578aad9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:42.022987 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8b9bb-548b-44fe-b02f-f98f578aad9a admin admin] PATH_INFO: `/v3/users/aff941a54eea4319ad5845e22eace9a1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:42.024217 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8b9bb-548b-44fe-b02f-f98f578aad9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aff941a54eea4319ad5845e22eace9a1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:42.026220 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8b9bb-548b-44fe-b02f-f98f578aad9a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:42.043818 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:42.084088 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dad8b9bb-548b-44fe-b02f-f98f578aad9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aff941a54eea4319ad5845e22eace9a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:42.084705 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dad8b9bb-548b-44fe-b02f-f98f578aad9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aff941a54eea4319ad5845e22eace9a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:42.091580 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dad8b9bb-548b-44fe-b02f-f98f578aad9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aff941a54eea4319ad5845e22eace9a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:42.108050 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dad8b9bb-548b-44fe-b02f-f98f578aad9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aff941a54eea4319ad5845e22eace9a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:42.108876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dad8b9bb-548b-44fe-b02f-f98f578aad9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aff941a54eea4319ad5845e22eace9a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:42.128200 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1439/2888] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:30:41 2022] DELETE /identity/v3/users/aff941a54eea4319ad5845e22eace9a1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:42.418022 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:42.419563 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:42.434291 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82437df9-7a58-45d1-96c9-e330ac4e81ea admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:42.486159 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:42.492999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444d739-a480-4c47-91e5-1c3b69ceeeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:42.493677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444d739-a480-4c47-91e5-1c3b69ceeeea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:42.494080 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444d739-a480-4c47-91e5-1c3b69ceeeea admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:42.494362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444d739-a480-4c47-91e5-1c3b69ceeeea admin admin] PATH_INFO: `/v3/projects/c0ef9bb471e54c8db060f600df849315` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:42.495652 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444d739-a480-4c47-91e5-1c3b69ceeeea admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ef9bb471e54c8db060f600df849315)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:42.498513 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444d739-a480-4c47-91e5-1c3b69ceeeea admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:42.547304 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3444d739-a480-4c47-91e5-1c3b69ceeeea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ef9bb471e54c8db060f600df849315', 'request_id': 'req-3444d739-a480-4c47-91e5-1c3b69ceeeea'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:42.553497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3444d739-a480-4c47-91e5-1c3b69ceeeea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ef9bb471e54c8db060f600df849315', 'request_id': 'req-3444d739-a480-4c47-91e5-1c3b69ceeeea'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:42.560070 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3444d739-a480-4c47-91e5-1c3b69ceeeea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ef9bb471e54c8db060f600df849315', 'request_id': 'req-3444d739-a480-4c47-91e5-1c3b69ceeeea'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:42.574544 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3444d739-a480-4c47-91e5-1c3b69ceeeea admin admin] The token cache is being invalidate because project c0ef9bb471e54c8db060f600df849315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:30:42.574952 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3444d739-a480-4c47-91e5-1c3b69ceeeea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:42.575595 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3444d739-a480-4c47-91e5-1c3b69ceeeea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:42.577150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1450/2889] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:30:42 2022] DELETE /identity/v3/projects/c0ef9bb471e54c8db060f600df849315 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:42.582175 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387ac57-a53d-4df3-8031-d6db4793091f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:42.582700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387ac57-a53d-4df3-8031-d6db4793091f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:42.582700 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387ac57-a53d-4df3-8031-d6db4793091f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:42.603991 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8387ac57-a53d-4df3-8031-d6db4793091f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:42.605256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8387ac57-a53d-4df3-8031-d6db4793091f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:42.674654 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1440/2890] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:42.682642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a4df6-470c-4c5f-b6a2-5a380d601ce9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:42.682927 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a4df6-470c-4c5f-b6a2-5a380d601ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:42.683197 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a4df6-470c-4c5f-b6a2-5a380d601ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:42.704990 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7b9a4df6-470c-4c5f-b6a2-5a380d601ce9 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:42.706207 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7b9a4df6-470c-4c5f-b6a2-5a380d601ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:42.715150 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1451/2891] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:42.719680 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387ac57-a53d-4df3-8031-d6db4793091f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:42.729960 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387ac57-a53d-4df3-8031-d6db4793091f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:42.734976 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e508a-8465-46d8-85de-342599cf9793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:42.735582 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e508a-8465-46d8-85de-342599cf9793 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:42.735857 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e508a-8465-46d8-85de-342599cf9793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:42.736128 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e508a-8465-46d8-85de-342599cf9793 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:42.736844 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e508a-8465-46d8-85de-342599cf9793 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:30:42.737129 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e508a-8465-46d8-85de-342599cf9793 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:42.738741 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e508a-8465-46d8-85de-342599cf9793 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:42.750893 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1441/2892] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:30:42.755419 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a4df6-470c-4c5f-b6a2-5a380d601ce9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:42.765359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a4df6-470c-4c5f-b6a2-5a380d601ce9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:42.770677 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f7275-a518-4b59-8470-6ae1a3f3ee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:42.771280 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f7275-a518-4b59-8470-6ae1a3f3ee9f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:42.771557 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f7275-a518-4b59-8470-6ae1a3f3ee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:42.771800 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f7275-a518-4b59-8470-6ae1a3f3ee9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:42.772451 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f7275-a518-4b59-8470-6ae1a3f3ee9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:42.774408 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f7275-a518-4b59-8470-6ae1a3f3ee9f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:42.806602 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1452/2893] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:30:42 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:42.810932 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e508a-8465-46d8-85de-342599cf9793 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:42.821676 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e508a-8465-46d8-85de-342599cf9793 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:42.878725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af22cce-3e6c-4e44-8d2a-458442acc3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:42.879252 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af22cce-3e6c-4e44-8d2a-458442acc3cb admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:42.879534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af22cce-3e6c-4e44-8d2a-458442acc3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:42.879799 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af22cce-3e6c-4e44-8d2a-458442acc3cb admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:42.880441 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af22cce-3e6c-4e44-8d2a-458442acc3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:42.882408 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af22cce-3e6c-4e44-8d2a-458442acc3cb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:42.905162 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1442/2894] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:30:42 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:42.909497 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f7275-a518-4b59-8470-6ae1a3f3ee9f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:42.920969 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f7275-a518-4b59-8470-6ae1a3f3ee9f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:42.927062 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b43d2-10aa-4aae-90b7-95464b24ce34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:42.927744 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b43d2-10aa-4aae-90b7-95464b24ce34 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:42.927961 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b43d2-10aa-4aae-90b7-95464b24ce34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:42.928271 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b43d2-10aa-4aae-90b7-95464b24ce34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:42.929000 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b43d2-10aa-4aae-90b7-95464b24ce34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:42.930681 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b43d2-10aa-4aae-90b7-95464b24ce34 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:42.941259 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1453/2895] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:42.945534 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af22cce-3e6c-4e44-8d2a-458442acc3cb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:42.956509 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af22cce-3e6c-4e44-8d2a-458442acc3cb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:42.962946 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231578f-3b9d-4fff-a91c-87b638eddc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:42.963548 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231578f-3b9d-4fff-a91c-87b638eddc18 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:42.963825 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231578f-3b9d-4fff-a91c-87b638eddc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:42.964094 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231578f-3b9d-4fff-a91c-87b638eddc18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:42.964783 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231578f-3b9d-4fff-a91c-87b638eddc18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:42.966530 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231578f-3b9d-4fff-a91c-87b638eddc18 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:42.971930 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:42.977593 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1443/2896] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:42.981915 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b43d2-10aa-4aae-90b7-95464b24ce34 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:42.992189 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b43d2-10aa-4aae-90b7-95464b24ce34 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:42.996934 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f46e6-faef-4b85-9b51-d784f80a537c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:42.997584 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f46e6-faef-4b85-9b51-d784f80a537c admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:42.997858 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f46e6-faef-4b85-9b51-d784f80a537c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:42.998131 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f46e6-faef-4b85-9b51-d784f80a537c admin admin] PATH_INFO: `/v3/projects/a4e0b2e482cc4dc396255c1f814403ad/users/5f6e47aaae984778abee1cdf70e890ec/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:43.014362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f46e6-faef-4b85-9b51-d784f80a537c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e0b2e482cc4dc396255c1f814403ad, user_id=5f6e47aaae984778abee1cdf70e890ec, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:43.017081 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f46e6-faef-4b85-9b51-d784f80a537c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:43.032255 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1454/2897] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:42 2022] PUT /identity/v3/projects/a4e0b2e482cc4dc396255c1f814403ad/users/5f6e47aaae984778abee1cdf70e890ec/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:43.044600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3a009-04b2-4957-b4b4-6680212f2303 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:43.044942 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3a009-04b2-4957-b4b4-6680212f2303 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:43.045218 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3a009-04b2-4957-b4b4-6680212f2303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:43.075152 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-68d3a009-04b2-4957-b4b4-6680212f2303 None None] MFA Rules not processed for user `5f6e47aaae984778abee1cdf70e890ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:43.076368 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-68d3a009-04b2-4957-b4b4-6680212f2303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:43.149838 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1444/2898] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:43.154620 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f46e6-faef-4b85-9b51-d784f80a537c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:43.217303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f46e6-faef-4b85-9b51-d784f80a537c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:43.223018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef781586-2e71-4167-a734-b46ad1e1d406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:43.223615 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef781586-2e71-4167-a734-b46ad1e1d406 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:43.223880 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef781586-2e71-4167-a734-b46ad1e1d406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:43.224149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef781586-2e71-4167-a734-b46ad1e1d406 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:43.224820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef781586-2e71-4167-a734-b46ad1e1d406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:43.226723 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef781586-2e71-4167-a734-b46ad1e1d406 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:43.262905 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1455/2899] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:30:43 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:43.267136 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3a009-04b2-4957-b4b4-6680212f2303 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:43.279876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3a009-04b2-4957-b4b4-6680212f2303 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:43.339894 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab0eb6-bdfa-4151-86c5-e952f6e4fc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:43.340494 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab0eb6-bdfa-4151-86c5-e952f6e4fc8c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:43.340798 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab0eb6-bdfa-4151-86c5-e952f6e4fc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:43.341065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab0eb6-bdfa-4151-86c5-e952f6e4fc8c admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:43.341693 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab0eb6-bdfa-4151-86c5-e952f6e4fc8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:43.343633 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab0eb6-bdfa-4151-86c5-e952f6e4fc8c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:43.366630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1445/2900] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:30:43 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:43.370854 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef781586-2e71-4167-a734-b46ad1e1d406 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:43.381328 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef781586-2e71-4167-a734-b46ad1e1d406 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:43.386364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524581e-ed37-462d-a8bf-dabafb722cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:43.386966 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524581e-ed37-462d-a8bf-dabafb722cdf admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:43.387244 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524581e-ed37-462d-a8bf-dabafb722cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:43.387511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524581e-ed37-462d-a8bf-dabafb722cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:43.388152 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4524581e-ed37-462d-a8bf-dabafb722cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:43.389853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4524581e-ed37-462d-a8bf-dabafb722cdf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:43.391971 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:43.402119 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1456/2901] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:43.406341 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab0eb6-bdfa-4151-86c5-e952f6e4fc8c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:43.416939 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab0eb6-bdfa-4151-86c5-e952f6e4fc8c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:43.421915 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7347585-9630-4b25-b1f5-e67f08648f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:43.422559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7347585-9630-4b25-b1f5-e67f08648f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:43.422830 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7347585-9630-4b25-b1f5-e67f08648f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:43.423096 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7347585-9630-4b25-b1f5-e67f08648f74 admin admin] PATH_INFO: `/v3/projects/4e815291d3f547d79bb18ea24a100268/users/6e0614f077584033b351705f2cfcf98c/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:43.439760 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7347585-9630-4b25-b1f5-e67f08648f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e815291d3f547d79bb18ea24a100268, user_id=6e0614f077584033b351705f2cfcf98c, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:43.442425 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7347585-9630-4b25-b1f5-e67f08648f74 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:43.457811 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1446/2902] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:43 2022] PUT /identity/v3/projects/4e815291d3f547d79bb18ea24a100268/users/6e0614f077584033b351705f2cfcf98c/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:43.462056 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524581e-ed37-462d-a8bf-dabafb722cdf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:43.472368 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524581e-ed37-462d-a8bf-dabafb722cdf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:43.535376 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b5e15-a3ff-46af-ab6f-6ae5c7c7c391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:43.535999 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b5e15-a3ff-46af-ab6f-6ae5c7c7c391 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:43.536276 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b5e15-a3ff-46af-ab6f-6ae5c7c7c391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:43.536987 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b5e15-a3ff-46af-ab6f-6ae5c7c7c391 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:43.537755 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b5e15-a3ff-46af-ab6f-6ae5c7c7c391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:43.539361 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b5e15-a3ff-46af-ab6f-6ae5c7c7c391 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:43.550637 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1457/2903] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:43.555118 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7347585-9630-4b25-b1f5-e67f08648f74 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:43.566241 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7347585-9630-4b25-b1f5-e67f08648f74 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:43.571060 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc7ebe-f697-41ff-9139-973c0fa27cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:43.571717 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc7ebe-f697-41ff-9139-973c0fa27cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:43.571995 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc7ebe-f697-41ff-9139-973c0fa27cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:43.572267 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc7ebe-f697-41ff-9139-973c0fa27cf4 admin admin] PATH_INFO: `/v3/projects/4e815291d3f547d79bb18ea24a100268/users/6e0614f077584033b351705f2cfcf98c/roles/3b30504991194002b41f8587d0851c10` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:43.574355 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc7ebe-f697-41ff-9139-973c0fa27cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e815291d3f547d79bb18ea24a100268, user_id=6e0614f077584033b351705f2cfcf98c, role_id=3b30504991194002b41f8587d0851c10)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:43.577183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc7ebe-f697-41ff-9139-973c0fa27cf4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:43.588628 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1447/2904] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:43 2022] PUT /identity/v3/projects/4e815291d3f547d79bb18ea24a100268/users/6e0614f077584033b351705f2cfcf98c/roles/3b30504991194002b41f8587d0851c10 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:43.599264 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef566da-edb7-4b8b-b44a-f00d02e0b4d3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:43.599572 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef566da-edb7-4b8b-b44a-f00d02e0b4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:43.599819 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef566da-edb7-4b8b-b44a-f00d02e0b4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:43.627631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8ef566da-edb7-4b8b-b44a-f00d02e0b4d3 None None] MFA Rules not processed for user `6e0614f077584033b351705f2cfcf98c`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:43.628874 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8ef566da-edb7-4b8b-b44a-f00d02e0b4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:43.712879 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1458/2905] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:43.723974 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc7ebe-f697-41ff-9139-973c0fa27cf4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:43.725865 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-84bc7ebe-f697-41ff-9139-973c0fa27cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:43.775376 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:43.788980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc7ebe-f697-41ff-9139-973c0fa27cf4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:43.792743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa49c91-0792-4fa7-824e-3887fad60b20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:43.793349 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa49c91-0792-4fa7-824e-3887fad60b20 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:43.793624 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa49c91-0792-4fa7-824e-3887fad60b20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:43.793891 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa49c91-0792-4fa7-824e-3887fad60b20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:43.803121 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa49c91-0792-4fa7-824e-3887fad60b20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:43.804559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa49c91-0792-4fa7-824e-3887fad60b20 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:43.814321 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1448/2906] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:30:44.242166 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef566da-edb7-4b8b-b44a-f00d02e0b4d3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:44.306374 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef566da-edb7-4b8b-b44a-f00d02e0b4d3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:44.313491 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:44.314149 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:44.314429 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:44.314699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] PATH_INFO: `/v3/users/5f6e47aaae984778abee1cdf70e890ec` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:44.315928 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f6e47aaae984778abee1cdf70e890ec)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:44.317936 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:44.340115 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:44.370531 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f6e47aaae984778abee1cdf70e890ec'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:44.371179 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f6e47aaae984778abee1cdf70e890ec'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:44.377940 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f6e47aaae984778abee1cdf70e890ec'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:44.389962 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f6e47aaae984778abee1cdf70e890ec'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:44.390607 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f6e47aaae984778abee1cdf70e890ec'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:44.407627 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1459/2907] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:30:44 2022] DELETE /identity/v3/users/5f6e47aaae984778abee1cdf70e890ec => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:44.412822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a22e1-99e6-41a5-a38b-9d9dfb389e7c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:44.413109 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a22e1-99e6-41a5-a38b-9d9dfb389e7c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:44.413374 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a22e1-99e6-41a5-a38b-9d9dfb389e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:44.435192 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5f7a22e1-99e6-41a5-a38b-9d9dfb389e7c None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:44.436434 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5f7a22e1-99e6-41a5-a38b-9d9dfb389e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:44.494493 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1449/2908] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:44.504359 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:44.505832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:44.565346 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43797d5a-be17-44d9-a022-5504da06bfdd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:44.569697 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7bf0f-f431-485e-bfdc-a06577a8d794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:44.570283 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7bf0f-f431-485e-bfdc-a06577a8d794 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:44.570558 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7bf0f-f431-485e-bfdc-a06577a8d794 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:44.570850 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7bf0f-f431-485e-bfdc-a06577a8d794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:44.581018 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7bf0f-f431-485e-bfdc-a06577a8d794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:44.582471 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7bf0f-f431-485e-bfdc-a06577a8d794 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:44.592596 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1460/2909] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:30:44.877384 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a22e1-99e6-41a5-a38b-9d9dfb389e7c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:44.878896 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5f7a22e1-99e6-41a5-a38b-9d9dfb389e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:44.895080 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a22e1-99e6-41a5-a38b-9d9dfb389e7c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:44.900074 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:44.900730 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:44.901009 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:44.901284 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] PATH_INFO: `/v3/projects/a4e0b2e482cc4dc396255c1f814403ad` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:44.902437 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e0b2e482cc4dc396255c1f814403ad)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:44.904551 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:44.954947 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e0b2e482cc4dc396255c1f814403ad', 'request_id': 'req-003d80d6-d8c3-423d-a536-3dee7d13310a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:44.964600 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e0b2e482cc4dc396255c1f814403ad', 'request_id': 'req-003d80d6-d8c3-423d-a536-3dee7d13310a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:44.971234 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e0b2e482cc4dc396255c1f814403ad', 'request_id': 'req-003d80d6-d8c3-423d-a536-3dee7d13310a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:44.982899 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] The token cache is being invalidate because project a4e0b2e482cc4dc396255c1f814403ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:30:44.983195 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:44.984031 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:44.985759 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1450/2910] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:30:44 2022] DELETE /identity/v3/projects/a4e0b2e482cc4dc396255c1f814403ad => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:44.989904 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7bf0f-f431-485e-bfdc-a06577a8d794 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:45.000132 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7bf0f-f431-485e-bfdc-a06577a8d794 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:45.064612 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fd0dd-54e3-4bb2-8608-84570524c66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:45.065282 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fd0dd-54e3-4bb2-8608-84570524c66a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:45.065564 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fd0dd-54e3-4bb2-8608-84570524c66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:45.065832 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fd0dd-54e3-4bb2-8608-84570524c66a admin admin] PATH_INFO: `/v3/users/6e0614f077584033b351705f2cfcf98c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:45.067014 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fd0dd-54e3-4bb2-8608-84570524c66a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0614f077584033b351705f2cfcf98c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:45.068993 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fd0dd-54e3-4bb2-8608-84570524c66a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:45.124598 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:45.127853 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c5fd0dd-54e3-4bb2-8608-84570524c66a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0614f077584033b351705f2cfcf98c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:45.128740 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c5fd0dd-54e3-4bb2-8608-84570524c66a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0614f077584033b351705f2cfcf98c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:45.138118 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c5fd0dd-54e3-4bb2-8608-84570524c66a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0614f077584033b351705f2cfcf98c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:45.149897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c5fd0dd-54e3-4bb2-8608-84570524c66a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0614f077584033b351705f2cfcf98c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:45.150548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9c5fd0dd-54e3-4bb2-8608-84570524c66a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0614f077584033b351705f2cfcf98c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:45.164831 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1461/2911] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:30:44 2022] DELETE /identity/v3/users/6e0614f077584033b351705f2cfcf98c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:45.455653 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:45.457102 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:45.470767 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d80d6-d8c3-423d-a536-3dee7d13310a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:45.532051 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92709a4-4661-461f-9666-d1992f81d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:45.532607 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92709a4-4661-461f-9666-d1992f81d453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:45.532921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92709a4-4661-461f-9666-d1992f81d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:45.533196 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92709a4-4661-461f-9666-d1992f81d453 admin admin] PATH_INFO: `/v3/projects/4e815291d3f547d79bb18ea24a100268` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:45.534368 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92709a4-4661-461f-9666-d1992f81d453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e815291d3f547d79bb18ea24a100268)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:45.536301 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92709a4-4661-461f-9666-d1992f81d453 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:45.548264 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:45.589040 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e92709a4-4661-461f-9666-d1992f81d453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e815291d3f547d79bb18ea24a100268', 'request_id': 'req-e92709a4-4661-461f-9666-d1992f81d453'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:45.599103 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e92709a4-4661-461f-9666-d1992f81d453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e815291d3f547d79bb18ea24a100268', 'request_id': 'req-e92709a4-4661-461f-9666-d1992f81d453'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:45.605921 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e92709a4-4661-461f-9666-d1992f81d453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e815291d3f547d79bb18ea24a100268', 'request_id': 'req-e92709a4-4661-461f-9666-d1992f81d453'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:45.617163 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e92709a4-4661-461f-9666-d1992f81d453 admin admin] The token cache is being invalidate because project 4e815291d3f547d79bb18ea24a100268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:30:45.617454 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e92709a4-4661-461f-9666-d1992f81d453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:45.618197 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e92709a4-4661-461f-9666-d1992f81d453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:45.619831 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1451/2912] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:30:45 2022] DELETE /identity/v3/projects/4e815291d3f547d79bb18ea24a100268 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:30:45.625865 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b7d45-f215-4ceb-95da-46475f864bc0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:45.626399 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b7d45-f215-4ceb-95da-46475f864bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:45.626399 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b7d45-f215-4ceb-95da-46475f864bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:45.647077 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-663b7d45-f215-4ceb-95da-46475f864bc0 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:45.648292 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-663b7d45-f215-4ceb-95da-46475f864bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:45.718728 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1462/2913] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:45.727322 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26adaaa-0454-4a71-8398-21dac7194dbf None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:45.727606 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26adaaa-0454-4a71-8398-21dac7194dbf None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:45.727884 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26adaaa-0454-4a71-8398-21dac7194dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:45.749610 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e26adaaa-0454-4a71-8398-21dac7194dbf None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:45.750836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e26adaaa-0454-4a71-8398-21dac7194dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:45.760000 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1452/2914] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:45.766174 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b7d45-f215-4ceb-95da-46475f864bc0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:45.776454 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b7d45-f215-4ceb-95da-46475f864bc0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:45.781403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b4368-b75f-404a-9dbe-943867af13f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:45.782007 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b4368-b75f-404a-9dbe-943867af13f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:45.782370 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b4368-b75f-404a-9dbe-943867af13f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:45.782548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b4368-b75f-404a-9dbe-943867af13f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:45.783223 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b4368-b75f-404a-9dbe-943867af13f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:30:45.783518 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b4368-b75f-404a-9dbe-943867af13f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:45.785163 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b4368-b75f-404a-9dbe-943867af13f9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:45.797577 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1463/2915] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:30:45.801980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26adaaa-0454-4a71-8398-21dac7194dbf None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:45.811642 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26adaaa-0454-4a71-8398-21dac7194dbf None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:45.816504 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb81f13-52ba-4899-bcbf-971b1ebcc946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:45.817145 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb81f13-52ba-4899-bcbf-971b1ebcc946 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:45.817423 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb81f13-52ba-4899-bcbf-971b1ebcc946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:45.817692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb81f13-52ba-4899-bcbf-971b1ebcc946 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:45.818327 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb81f13-52ba-4899-bcbf-971b1ebcc946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:45.820241 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb81f13-52ba-4899-bcbf-971b1ebcc946 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:45.851440 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1453/2916] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:30:45 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 21:30:45.858617 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b4368-b75f-404a-9dbe-943867af13f9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:45.875071 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b4368-b75f-404a-9dbe-943867af13f9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:45.939076 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ece75-3e7d-497f-89ad-199e79c6199f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:45.939699 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ece75-3e7d-497f-89ad-199e79c6199f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:45.939980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ece75-3e7d-497f-89ad-199e79c6199f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:45.940254 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ece75-3e7d-497f-89ad-199e79c6199f admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:45.940926 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ece75-3e7d-497f-89ad-199e79c6199f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:45.942893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ece75-3e7d-497f-89ad-199e79c6199f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:45.968048 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1464/2917] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:30:45 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:45.972880 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb81f13-52ba-4899-bcbf-971b1ebcc946 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:45.984883 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb81f13-52ba-4899-bcbf-971b1ebcc946 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:45.989836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a41f4-ae2c-46fd-b878-2026ca8390de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:45.990432 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a41f4-ae2c-46fd-b878-2026ca8390de admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:45.990704 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a41f4-ae2c-46fd-b878-2026ca8390de admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:45.990968 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a41f4-ae2c-46fd-b878-2026ca8390de admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:45.991603 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a41f4-ae2c-46fd-b878-2026ca8390de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:45.993267 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a41f4-ae2c-46fd-b878-2026ca8390de admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:46.003211 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1454/2918] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:46.007403 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ece75-3e7d-497f-89ad-199e79c6199f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:46.017873 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ece75-3e7d-497f-89ad-199e79c6199f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:46.023085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028e325-5938-47d7-9977-a583159d8b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:46.023688 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028e325-5938-47d7-9977-a583159d8b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:46.024031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028e325-5938-47d7-9977-a583159d8b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:46.024303 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028e325-5938-47d7-9977-a583159d8b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:46.024979 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028e325-5938-47d7-9977-a583159d8b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:46.026586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028e325-5938-47d7-9977-a583159d8b04 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:46.038326 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1465/2919] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:46.044235 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a41f4-ae2c-46fd-b878-2026ca8390de admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:46.055822 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a41f4-ae2c-46fd-b878-2026ca8390de admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:46.061778 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97033fc7-3f46-415b-b59b-9a543d751332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:46.062424 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97033fc7-3f46-415b-b59b-9a543d751332 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:46.062692 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97033fc7-3f46-415b-b59b-9a543d751332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:46.062956 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97033fc7-3f46-415b-b59b-9a543d751332 admin admin] PATH_INFO: `/v3/projects/635e5e33435a42a3bfa8e9b7c64f10ca/users/f58f750a5afd42f3b53d6c4175deb7fc/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:46.080503 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97033fc7-3f46-415b-b59b-9a543d751332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=635e5e33435a42a3bfa8e9b7c64f10ca, user_id=f58f750a5afd42f3b53d6c4175deb7fc, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:46.083180 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97033fc7-3f46-415b-b59b-9a543d751332 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:46.103436 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1455/2920] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:46 2022] PUT /identity/v3/projects/635e5e33435a42a3bfa8e9b7c64f10ca/users/f58f750a5afd42f3b53d6c4175deb7fc/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 21:30:46.120829 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27799d-814e-4a36-9841-e3ba89cc09b9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:46.121112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27799d-814e-4a36-9841-e3ba89cc09b9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:46.121397 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27799d-814e-4a36-9841-e3ba89cc09b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:46.151156 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ef27799d-814e-4a36-9841-e3ba89cc09b9 None None] MFA Rules not processed for user `f58f750a5afd42f3b53d6c4175deb7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:46.152362 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ef27799d-814e-4a36-9841-e3ba89cc09b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:46.165107 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:46.219580 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1466/2921] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:46.230836 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97033fc7-3f46-415b-b59b-9a543d751332 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:46.232789 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-97033fc7-3f46-415b-b59b-9a543d751332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:46.297807 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97033fc7-3f46-415b-b59b-9a543d751332 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:46.304311 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949cd027-0b07-4e56-be2b-9fb0fbd2eac2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:46.305396 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cd027-0b07-4e56-be2b-9fb0fbd2eac2 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:46.305682 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cd027-0b07-4e56-be2b-9fb0fbd2eac2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:46.305950 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cd027-0b07-4e56-be2b-9fb0fbd2eac2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:46.315721 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949cd027-0b07-4e56-be2b-9fb0fbd2eac2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:46.317223 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949cd027-0b07-4e56-be2b-9fb0fbd2eac2 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:46.326671 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1456/2922] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:30:49.416113 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27799d-814e-4a36-9841-e3ba89cc09b9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:49.483642 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27799d-814e-4a36-9841-e3ba89cc09b9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:49.489261 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:49.489925 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:49.490194 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:49.490479 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] PATH_INFO: `/v3/users/f58f750a5afd42f3b53d6c4175deb7fc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:49.491721 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f58f750a5afd42f3b53d6c4175deb7fc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:49.493734 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:49.548512 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f58f750a5afd42f3b53d6c4175deb7fc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:49.549181 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f58f750a5afd42f3b53d6c4175deb7fc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:49.555909 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f58f750a5afd42f3b53d6c4175deb7fc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:49.567535 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f58f750a5afd42f3b53d6c4175deb7fc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:49.568169 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f58f750a5afd42f3b53d6c4175deb7fc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:49.582227 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1467/2923] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:30:49 2022] DELETE /identity/v3/users/f58f750a5afd42f3b53d6c4175deb7fc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:49.588876 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191187b-a91c-4342-af75-735b88a9c090 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:49.589381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191187b-a91c-4342-af75-735b88a9c090 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:49.589381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191187b-a91c-4342-af75-735b88a9c090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:49.609772 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0191187b-a91c-4342-af75-735b88a9c090 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:49.611549 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0191187b-a91c-4342-af75-735b88a9c090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:49.649477 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:49.679065 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1457/2924] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:49.690460 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:49.692136 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:49.734556 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:49.754782 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91bc03-3142-49e6-be95-d301479908d8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:49.759756 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8da88-d4f2-4e54-993d-729ac3d2ee8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:49.760625 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8da88-d4f2-4e54-993d-729ac3d2ee8e service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:49.760938 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8da88-d4f2-4e54-993d-729ac3d2ee8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:49.761200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8da88-d4f2-4e54-993d-729ac3d2ee8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:49.772728 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8da88-d4f2-4e54-993d-729ac3d2ee8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:49.774198 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8da88-d4f2-4e54-993d-729ac3d2ee8e service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:49.783998 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1468/2925] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:30:50.067725 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0191187b-a91c-4342-af75-735b88a9c090 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:50.069353 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0191187b-a91c-4342-af75-735b88a9c090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:50.084183 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0191187b-a91c-4342-af75-735b88a9c090 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:50.089508 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8444e-2679-436b-8b89-d612a0911b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:50.090381 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8444e-2679-436b-8b89-d612a0911b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:50.090657 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8444e-2679-436b-8b89-d612a0911b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:50.090929 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8444e-2679-436b-8b89-d612a0911b2e admin admin] PATH_INFO: `/v3/projects/635e5e33435a42a3bfa8e9b7c64f10ca` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:50.092262 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8444e-2679-436b-8b89-d612a0911b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=635e5e33435a42a3bfa8e9b7c64f10ca)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:50.094232 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8444e-2679-436b-8b89-d612a0911b2e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:50.145482 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b4c8444e-2679-436b-8b89-d612a0911b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '635e5e33435a42a3bfa8e9b7c64f10ca', 'request_id': 'req-b4c8444e-2679-436b-8b89-d612a0911b2e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:50.155599 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b4c8444e-2679-436b-8b89-d612a0911b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '635e5e33435a42a3bfa8e9b7c64f10ca', 'request_id': 'req-b4c8444e-2679-436b-8b89-d612a0911b2e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:50.163281 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b4c8444e-2679-436b-8b89-d612a0911b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '635e5e33435a42a3bfa8e9b7c64f10ca', 'request_id': 'req-b4c8444e-2679-436b-8b89-d612a0911b2e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:50.174670 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b4c8444e-2679-436b-8b89-d612a0911b2e admin admin] The token cache is being invalidate because project 635e5e33435a42a3bfa8e9b7c64f10ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:30:50.174956 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b4c8444e-2679-436b-8b89-d612a0911b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:50.175673 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b4c8444e-2679-436b-8b89-d612a0911b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:30:50.177295 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1458/2926] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:30:50 2022] DELETE /identity/v3/projects/635e5e33435a42a3bfa8e9b7c64f10ca => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:50.182586 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069597b-7a57-44a2-979f-de54ec42183f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:50.182810 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069597b-7a57-44a2-979f-de54ec42183f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:50.183085 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069597b-7a57-44a2-979f-de54ec42183f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:50.203134 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2069597b-7a57-44a2-979f-de54ec42183f None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:50.204364 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2069597b-7a57-44a2-979f-de54ec42183f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:50.268508 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1469/2927] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:50.276910 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3f261-128d-4d3e-83d1-7a2924470355 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:50.277191 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3f261-128d-4d3e-83d1-7a2924470355 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:50.277466 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3f261-128d-4d3e-83d1-7a2924470355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:50.298879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-abc3f261-128d-4d3e-83d1-7a2924470355 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:50.300090 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-abc3f261-128d-4d3e-83d1-7a2924470355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:50.309141 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1459/2928] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 21:30:50.313510 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069597b-7a57-44a2-979f-de54ec42183f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:50.323679 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069597b-7a57-44a2-979f-de54ec42183f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:50.329293 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b5c7b-e4ed-4d96-8031-efafbbcd52fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:50.329897 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b5c7b-e4ed-4d96-8031-efafbbcd52fe admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:50.330173 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b5c7b-e4ed-4d96-8031-efafbbcd52fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:50.330439 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b5c7b-e4ed-4d96-8031-efafbbcd52fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:50.331089 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b5c7b-e4ed-4d96-8031-efafbbcd52fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 21:30:50.331385 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b5c7b-e4ed-4d96-8031-efafbbcd52fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:50.333096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b5c7b-e4ed-4d96-8031-efafbbcd52fe admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:50.345666 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1470/2929] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Sep 6 21:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 21:30:50.350184 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3f261-128d-4d3e-83d1-7a2924470355 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:50.360504 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3f261-128d-4d3e-83d1-7a2924470355 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:50.365672 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b91aa-7ee9-4b75-9d3f-b91033d8a377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:50.366276 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b91aa-7ee9-4b75-9d3f-b91033d8a377 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:50.366550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b91aa-7ee9-4b75-9d3f-b91033d8a377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:50.366817 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b91aa-7ee9-4b75-9d3f-b91033d8a377 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:50.367444 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b91aa-7ee9-4b75-9d3f-b91033d8a377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:50.369389 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b91aa-7ee9-4b75-9d3f-b91033d8a377 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:50.401440 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1460/2930] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Sep 6 21:30:50 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:50.402714 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b5c7b-e4ed-4d96-8031-efafbbcd52fe admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:50.413091 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b5c7b-e4ed-4d96-8031-efafbbcd52fe admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:50.474837 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda5263-611c-48d8-b4ad-042c713527ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:50.475523 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda5263-611c-48d8-b4ad-042c713527ca admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:50.475631 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda5263-611c-48d8-b4ad-042c713527ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:50.475893 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda5263-611c-48d8-b4ad-042c713527ca admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:50.476526 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda5263-611c-48d8-b4ad-042c713527ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:50.478505 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda5263-611c-48d8-b4ad-042c713527ca admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:50.502487 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1471/2931] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Sep 6 21:30:50 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 21:30:50.507526 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b91aa-7ee9-4b75-9d3f-b91033d8a377 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:50.520550 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b91aa-7ee9-4b75-9d3f-b91033d8a377 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:50.526340 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa3b7c-f785-4f2a-aa18-5f93dde3ba07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:50.526935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa3b7c-f785-4f2a-aa18-5f93dde3ba07 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:50.527202 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa3b7c-f785-4f2a-aa18-5f93dde3ba07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:50.527467 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa3b7c-f785-4f2a-aa18-5f93dde3ba07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:50.528095 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa3b7c-f785-4f2a-aa18-5f93dde3ba07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:50.529793 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa3b7c-f785-4f2a-aa18-5f93dde3ba07 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:50.541147 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1461/2932] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:50.545592 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda5263-611c-48d8-b4ad-042c713527ca admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:50.556338 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda5263-611c-48d8-b4ad-042c713527ca admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:50.561375 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e074b3d-d0b1-4ab6-b77c-611e8663be3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:50.561977 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e074b3d-d0b1-4ab6-b77c-611e8663be3e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:50.562253 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e074b3d-d0b1-4ab6-b77c-611e8663be3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:50.562520 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e074b3d-d0b1-4ab6-b77c-611e8663be3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:50.563158 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e074b3d-d0b1-4ab6-b77c-611e8663be3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:50.564804 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e074b3d-d0b1-4ab6-b77c-611e8663be3e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:50.575306 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1472/2933] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Sep 6 21:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 21:30:50.579630 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa3b7c-f785-4f2a-aa18-5f93dde3ba07 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:50.590231 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa3b7c-f785-4f2a-aa18-5f93dde3ba07 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:50.595573 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a8ee2-f6e6-4a71-a5e8-061528a8c6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:50.596273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a8ee2-f6e6-4a71-a5e8-061528a8c6bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:50.596553 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a8ee2-f6e6-4a71-a5e8-061528a8c6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:50.596879 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a8ee2-f6e6-4a71-a5e8-061528a8c6bf admin admin] PATH_INFO: `/v3/projects/3bbafc413278481588e1891ebf823087/users/11b0c77373964beb975870b4b086f079/roles/180edbb8a9e74555af89d6a7639f819a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:50.612743 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a8ee2-f6e6-4a71-a5e8-061528a8c6bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bbafc413278481588e1891ebf823087, user_id=11b0c77373964beb975870b4b086f079, role_id=180edbb8a9e74555af89d6a7639f819a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:50.615446 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a8ee2-f6e6-4a71-a5e8-061528a8c6bf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:50.629845 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1462/2934] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Sep 6 21:30:50 2022] PUT /identity/v3/projects/3bbafc413278481588e1891ebf823087/users/11b0c77373964beb975870b4b086f079/roles/180edbb8a9e74555af89d6a7639f819a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:30:50.640901 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095c92d-94da-4716-99b5-2e86b2886e2c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:50.641210 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095c92d-94da-4716-99b5-2e86b2886e2c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:50.641477 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095c92d-94da-4716-99b5-2e86b2886e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:50.671096 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7095c92d-94da-4716-99b5-2e86b2886e2c None None] MFA Rules not processed for user `11b0c77373964beb975870b4b086f079`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:30:50.672297 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7095c92d-94da-4716-99b5-2e86b2886e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:50.705112 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:50.736907 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1473/2935] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:30:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:30:50.749173 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a8ee2-f6e6-4a71-a5e8-061528a8c6bf admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:30:50.752047 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0b0a8ee2-f6e6-4a71-a5e8-061528a8c6bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:30:50.816748 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a8ee2-f6e6-4a71-a5e8-061528a8c6bf admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:30:50.820545 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925bc9f-7999-402e-bbcb-4e9bc158839f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ae3b9997d4dc39c3d67ca40bbc566', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:30:50.821193 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925bc9f-7999-402e-bbcb-4e9bc158839f service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:30:50.821448 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925bc9f-7999-402e-bbcb-4e9bc158839f service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:30:50.821754 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925bc9f-7999-402e-bbcb-4e9bc158839f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:30:50.831474 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925bc9f-7999-402e-bbcb-4e9bc158839f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:30:50.833022 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925bc9f-7999-402e-bbcb-4e9bc158839f service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:30:50.837697 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:30:50.842980 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1463/2936] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:30:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:31:00.734200 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=12 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:31:11.408114 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095c92d-94da-4716-99b5-2e86b2886e2c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:31:11.476142 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095c92d-94da-4716-99b5-2e86b2886e2c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:31:11.481103 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:31:11.481760 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:31:11.482039 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:31:11.482310 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] PATH_INFO: `/v3/users/11b0c77373964beb975870b4b086f079` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:31:11.483481 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b0c77373964beb975870b4b086f079)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:31:11.485450 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:31:11.538820 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b0c77373964beb975870b4b086f079'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:31:11.539614 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b0c77373964beb975870b4b086f079'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:31:11.546872 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b0c77373964beb975870b4b086f079'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:31:11.561349 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b0c77373964beb975870b4b086f079'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:31:11.562092 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b0c77373964beb975870b4b086f079'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:31:11.577031 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1474/2937] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Sep 6 21:31:11 2022] DELETE /identity/v3/users/11b0c77373964beb975870b4b086f079 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:31:11.581666 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467eb28-19d5-4698-8a55-3b62af708624 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:31:11.582228 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467eb28-19d5-4698-8a55-3b62af708624 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:31:11.582346 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467eb28-19d5-4698-8a55-3b62af708624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:31:11.602359 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a467eb28-19d5-4698-8a55-3b62af708624 None None] MFA Rules not processed for user `42a19c5d3ad549648071d18e7300d6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 21:31:11.603559 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a467eb28-19d5-4698-8a55-3b62af708624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:31:11.671561 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1464/2938] 10.222.0.79 () {58 vars in 958 bytes} [Tue Sep 6 21:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 21:31:11.684152 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:31:11.687140 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:31:11.751747 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591760ef-1844-411a-bdb6-8bd3c54633f1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:31:11.756100 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1849b-30fa-48b8-ae98-949d09e03374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5a45dec40844877a6b5b301fc44a110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:31:11.756766 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1849b-30fa-48b8-ae98-949d09e03374 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:31:11.757045 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1849b-30fa-48b8-ae98-949d09e03374 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:31:11.757316 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1849b-30fa-48b8-ae98-949d09e03374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:31:11.767649 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1849b-30fa-48b8-ae98-949d09e03374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:31:11.769170 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1849b-30fa-48b8-ae98-949d09e03374 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:31:11.779386 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1475/2939] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:31:12.059779 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467eb28-19d5-4698-8a55-3b62af708624 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:31:12.061346 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a467eb28-19d5-4698-8a55-3b62af708624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:31:12.076926 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467eb28-19d5-4698-8a55-3b62af708624 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:31:12.083496 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c57a4a-58be-470e-81a5-0d6a44520059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a19c5d3ad549648071d18e7300d6c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b4b2ee27141432e8ca02dac223fb495', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:31:12.084158 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c57a4a-58be-470e-81a5-0d6a44520059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:31:12.084438 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c57a4a-58be-470e-81a5-0d6a44520059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:31:12.084736 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c57a4a-58be-470e-81a5-0d6a44520059 admin admin] PATH_INFO: `/v3/projects/3bbafc413278481588e1891ebf823087` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:31:12.085935 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c57a4a-58be-470e-81a5-0d6a44520059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bbafc413278481588e1891ebf823087)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:31:12.087908 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c57a4a-58be-470e-81a5-0d6a44520059 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:31:12.142858 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54c57a4a-58be-470e-81a5-0d6a44520059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bbafc413278481588e1891ebf823087', 'request_id': 'req-54c57a4a-58be-470e-81a5-0d6a44520059'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:31:12.153134 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54c57a4a-58be-470e-81a5-0d6a44520059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bbafc413278481588e1891ebf823087', 'request_id': 'req-54c57a4a-58be-470e-81a5-0d6a44520059'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:31:12.160202 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54c57a4a-58be-470e-81a5-0d6a44520059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bbafc413278481588e1891ebf823087', 'request_id': 'req-54c57a4a-58be-470e-81a5-0d6a44520059'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:31:12.172273 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54c57a4a-58be-470e-81a5-0d6a44520059 admin admin] The token cache is being invalidate because project 3bbafc413278481588e1891ebf823087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 21:31:12.172562 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54c57a4a-58be-470e-81a5-0d6a44520059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:31:12.173256 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54c57a4a-58be-470e-81a5-0d6a44520059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 21:31:12.174853 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1465/2940] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Sep 6 21:31:12 2022] DELETE /identity/v3/projects/3bbafc413278481588e1891ebf823087 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 21:31:21.774809 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:31:22.169021 dv-cinder-749155-11-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 21:33:12.631037 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1849b-30fa-48b8-ae98-949d09e03374 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 21:33:12.632806 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dfc1849b-30fa-48b8-ae98-949d09e03374 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:33:12.796129 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1849b-30fa-48b8-ae98-949d09e03374 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 21:33:12.863452 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c638f3-94e8-4d85-a8fe-20993d835250 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c40b5ff21d1940749bf479cc936fc0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05af249ba0a047fe83aefb078e203e53', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 21:33:12.863980 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c638f3-94e8-4d85-a8fe-20993d835250 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 21:33:12.864260 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c638f3-94e8-4d85-a8fe-20993d835250 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 21:33:12.864527 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c638f3-94e8-4d85-a8fe-20993d835250 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 21:33:12.866548 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f4c638f3-94e8-4d85-a8fe-20993d835250 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 21:33:12.878679 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c638f3-94e8-4d85-a8fe-20993d835250 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 21:33:12.880299 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c638f3-94e8-4d85-a8fe-20993d835250 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 21:33:12.891430 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1476/2941] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Sep 6 21:33:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 21:33:22.886511 dv-cinder-749155-11-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}